Managing and Enforcing Automated DevOps of MQ Environments
Nastel is well known for producing some powerful GUIs (graphical user interfaces) for managing and monitoring IBM MQ but is there still a place for these as the world moves towards DevOps with automated continuous integration and continuous delivery?
Of course, the answer is yes! All the technology that Nastel has spent 25 years developing as the back end behind the GUIs can provide a secure and robust interface between the new open source tooling and the MQ environments, and the GUIs themselves can still be used as much more intuitive components in the automated software development lifecycle, thus reducing the chances of human error and reducing the cost of ownership.
The aim of an automated software development lifecycle is that the definitive source of the truth for the MQ configuration is held in a version control system, that all changes to it are recorded and audited and that the environment can be quickly and automatically be built from this as infrastructure as code. In this way a configuration can be built in one or more development environments, integrated into the version control system, with continuous integration, and then deployed into the test environments, then finally through pre-prod and into production. You can be confident that no manual changes to one environment have been missed in delivery to the next environment. As we move into a cloud-based utility computing world it becomes very useful and financially beneficial to only have environments for as long as they’re needed. We can spin up a test environment, deploy the configuration to it, run the tests and then shut the whole thing down and only pay for the infrastructure during the time when it’s needed.
So, what technologies are people using for this? Well, I’m finding that people are using versions of Git such as Github or Bitbucket for the version control system / source code control repository. For the continuous integration Jenkins is often used, and Ansible (free version or Ansible Tower) can be used to manage the deployment. Chef, Puppet and Terraform are other technologies that many choose as alternatives.
So, isn’t it enough to write lots of scripts using these open source technologies? Why would you need Nastel?
From the beginning (MQ V2) MQ has had a script command processor called MQSC which can define and alter all the objects within a queue manager. These scripts would form the basis of an automated deployment. However, we still need to design the environment which gets scripted, as well as defining the queue managers and associated objects (listener, command server, channel initiator etc).
Nastel Navigator provides a GUI in which a developer can design and unit test an MQ configuration. From here you can save the environment to a script which can be stored in a version control system and act as input to MQSC. With a Nastel agent on the target machine it can create the queue manager and associated objects as well as configuring it.
So, you can put the configuration definitions into the source code repository from Navigator, or write them directly as text. Jenkins can detect the updates and pull them from the repository and then pass them to Ansible to initiate the deployment to the next environment.
At this point Ansible can use the Nastel Navigator agent to create or update the new environment using the scripts. There is a lot of security functionality in Navigator which will ensure that developers can only update the objects which they are authorised to and so cannot affect the work or environments of other users or teams. This security and governance is critical to the DevOps dream becoming reality.
So, we’ve migrated a configuration from development to test, and similarly it can move from test to production etc in a robust automated way. So, is that enough? What happens if someone makes changes in the test environment? What if they bypass the tooling and log straight in to fix something in production? How would we detect that? How can we get that change reflected back into the source control repository, to ensure that the Git version remains the golden copy, the single source of the truth, and that if production fails we can always reproduce it?
To address this Navigator can monitor the environment. Firstly, it can receive MQ Configuration events which are generated when certain MQ actions are taken such as alter, create and delete. Secondly, Navigator generates its own Alter events. It constantly compares the MQ environment to the version that it has stored in its database and when it spots a difference it records it as an event.
You can then choose whether to roll back this change or to save it as a script for the version control system.
Finally, how do we know that it’s a valid configuration? How do we know that the design, whether created as a script or through the GUI, conforms to the company’s design standards? Will it actually perform in the expected way? Do all the queue managers have dead letter queues? Have all the listeners and channel initiators been defined? Do all the transmission queues exist that are referred to by the channels and remote queue definitions? Do the objects conform to your naming standards?
This is where Nastel AutoPilot comes in. AutoPilot is a real time performance monitor and typically comes with Navigator. AutoPilot can monitor any of the environments, whether that’s the development environment or the test or production ones, and highlight any issues such as these.
Whilst it is possible to build an automated software development lifecycle environment using open source tools, a lot of the effort can be reduced by adding Nastel’s Navigator and AutoPilot software to the mix, adding in its twenty five years of MQ and monitoring expertise to assure security, governance, and a lower cost of ownership.
I would be very keen to hear your comments, questions and experiences of DevOps with MQ.