During the past 7 business days I took part in a training about SIMATIC IT XHQ 4.0 software.
I participated in the Basic (21-22 Jan) and Advanced (25-29 Jan) training.

This training was given by the engineer Fabio Terasaka who’s a team lead at Chemtech having over 3 years of experience using and deploying XHQ in Brazil and internationally.

I decided to write this post so that people can get an overview of XHQ from a consultant/developer perspective.

I’m also excited about the endless possibilities XHQ has to offer when it comes to optimizing and applying intelligence to an enterprise.

What is XHQ?
For those who don’t know or who have never heard of XHQ, here goes a succinct description of it extracted from its official site:

SIMATIC IT XHQ Operations Intelligence product line aggregates, relates and presents operational and business data in real-time to improve enterprise performance. Through SIMATIC IT XHQ, you have a single coherent view of information, enabling a variety of solutions in real-time performance management and decision support. ^[1]

XQH extracts data from a variety of systems - as the production (PIMS), laboratory (LIMS) and plant-floor systems. XHQ unifies all the operational and management data in a single view, in real time, allowing you to take a snapshot, minute to minute or second to second, of all the enterprise.

XHQ can be integrated in the intranet or a website for operations management, integrating production data such as the use of raw material and equipment, stocks, as well as data related to the product (temperature, pressure, electrical current), quality and maintenance.

XHQ implements the concepts of Operational Dashboard and Management ^[2] by Key Performance Indicators (KPIs) ^[3].

XHQ is used in energy, petrochemical, and manufacturing industries to aggregate, draw relationships, and then graphically depict business and operational data.

XHQ Timeline
XHQ was created in 1996 by an American company called IndX Software Corp based on Aliso Viejo, California, USA.

In December 2003 Siemens expands its IT portfolio acquiring IndX ^[4].

In December 2009 Chemtech - A Siemens Company absorbs the company responsible for XHQ around the world ^[5].

XHQ Architecture
XHQ has a modular architecture as can be seen in the following picture:

XHQ Architecture Overview

Back-end Operational Systems
Comprised of databases and its respective connectors that give access to business real-time data: times series data (PHD, PI, OPC), real-time point data (Tags), relational databases (Oracle, MS-SQL), enterprise applications (SAP), etc.

Middle Tier
Comprised of XHQ set of servers. Each XHQ server plays a role in the system:

XHQ Enterprise Server manages the end-users views of data that are created by XHQ developers.

XHQ Solution Server has the Real-time Data Cache and the Relational Data Cache that removes the burden associated with backend data retrieval.

XHQ Alert Notification Server (XANS) is a subsystem of XHQ responsible for alerting end-users about any inconsistence existent in the system.

3rd Party Web Servers as IIS and Tomcat give end-users access to data processed by XHQ.

User Interface
Users can access XHQ processed data (Views) using PDAs, web browsers, etc.

Users also have access to View Statistics that is a kind of Google Analytics. It shows default reports about peak and average user count, user and view hits by month, user and view hits by week, view usage by user per day, view usage per day, etc. You can create your own analytics reports using custom SQL.

Starting with XHQ 4.0 there’s a separate application called Visual Composer that enables developers to create dynamic, high customizable data views. Visual Composer can use XHQ data collections as its data source.
Visual Composer focus in graphics/charts and tables/grids to show business strategic content.

XHQ behind the Curtains
XHQ does its job using a subscription model based on the client-server architecture. Clients are automatically notified of changes that occur on the monitored variables. For example, if a user uses a view that has 2 plant variables and their pool period (configured in the connector or on the variable itself) is set with 2 seconds, the user screen will automatically refresh (using Ajax) to show the new variable values at each 2 seconds. This is the so called real-time process management.

XHQ core is implemented in Java and uses a Java Applet that is loaded in the browser to present the data to the user.

XHQ makes extensive use of JavaScript to inject customization points into the software.

Servers configuration are kept in .properties files making it easy to edit.

Data presented to the user comes from “Collections” that use high performance data caches that are XHQ own local databases. You can use live data from the backend but it’s not advisable because of the overhead implied. The performance gains can be better verified when lots of users are using the same view.

Skills demanded by XHQ
To get XHQ up and running you’ll need the following skills:

SQL query skills. SQL is used all the time to retrieve the right data from the back-ends.

XML and XSLT skills. Both necessary to configure data points (Tags) in the system and to export data.

Previous software development skills using the .NET Framework or Java are important to develop extension points to XHQ.

JavaScript skills. Used to define custom system variables and client configuration.

HTML and CSS skills. Used to customize the user UI.

Web server administration using IIS and Tomcat is a plus when it comes to deploying the solution in the customer.

Computer network skills. Used to detected any problem between clients and servers.

Solid debugging skills related to the above mentioned technologies. If something goes wrong, you’ll need to check a lot of log files (there is one for each agent in the system).

XHQ Implementation
XHQ consultants/engineers are the guys responsible for studying the necessities of the customer interested in optimizing the enterprise.

The following are 10 basic steps used when XHQ is implemented as the choice for business optimization:

01 - XHQ is installed on client premises;
02 - Groups of users and use cases are defined;
03 - Connectors are created to access data sources scattered all over the enterprise;
04 - A solution model is defined;
05 - A navigation model is defined; 
06 - Views of data for different audiences and activities are built;
07 - System components and collections of data are linked for data retrieval;
08 - The solution is updated, tested and optimized;
09 - Steps 1 through 7 are iterated;
10 - Security is applied in the solution model through the use of roles/user groups.

XHQ Value as a RtPM Tool
XHQ aggregates value to your business as a RtPM (Real-time Process Management) tool:

Using XHQ, operational costs may decrease an average of 8% each year, while the production of high value products may increase 10.5%. This is because XHQ helps the management board in the decision taking process. ^[6]

The following are 10 basic reasons why XHQ aggregates value to the business:

01 - Directors and staff can take their decisions based on the same information;
02 - Response times are dramatically reduced;
03 - Information availability is made true from one area to another (and vice-versa);
04 - Interfaces between one area and another, usually managed by different teams and systems, can be closely monitored;
05 - User-friendly and self-explained process schematics simplify plan management;
06 - Reduced load on mission-critical systems: read only users can use only XHQ;
07 - Leverage of other investments: PIMS systems utilization is increased, and become mission-critical as well;
08 - Intangible gain: re-think strategies for company needs in terms of what information is
considered critical for business decisions;
09 - Integration with enterprise applications as SAP R/3, logistics is greatly improved by
watching supply and distribution movements;
10 - Transport can come and go (monitored) graphically.

XHQ Customers
XHQ is used throughout the world.

The following are some of the customers already using XHQ to optimize their business:

CSN, ExxonMobil, Chevron, Dow Chemical, Saudi Aramco among others.

Interested in optimizing your business?
If you’re looking for business optimization/intelligence, you can get in contact with Chemtech for more information.

References
[1] SIMATIC IT XHQ official site

[2] Business Performance Management

[3] Key Performance Indicator

[4] Siemens expands its IT portfolio in process industries (PDF file)

[5] Chemtech absorbs the company responsible for XHQ around the world

[6] Siemens of Brazil Press Information (in Portuguese)

[7] XHQ for Steel Mills Real Time Performance Management (PDF file)

[8] XHQ can gather information from the whole oil & gas production chain

[9] Chemtech enters into the IndX’s biggest XHQ project in Brazil

I had to install the Blogs portlet in Liferay.

Liferay is the all purpose portal framework that Chemtech uses to build its website.

The Liferay portal already deployed on production server is the 3.4.5 version. When I tried to add the Blogs portlet through the Add Content menu option I couldn’t find it.

Googling about Liferay’s Blogs portlet didn’t help me. The only positive clue I had was

Liferay Portal Administrator's Guide, Third Edition

(page 124) which has a section dedicated to the Blogs portlet.

I tried to understand why the Blogs portlet wasn’t available in the Add Content window:

Was it because the blogs portlet didn’t make it into the version 4.3.5 of the portal? The answer is no. The blogs portlet is available in version 4.3.5 (with limitations if compared to the Blogs portlet of today’s Liferay version that is currently 5.2.3).

After a little bit of more googling I found Development in the ext environment wiki article. I read in item 4 that you can turn portlets you want to deploy on/off by editing the file

\ext\ext-web\docroot\WEB-INF\liferay-portlet-ext.xml

Mine was located in

E:\chemsite\tomcat\webapps\lportal\WEB-INF\liferay-portlet-ext.xml

I did just that turning the Blogs portlet ON setting the <include> property to true:

<!--
    Liferay Portlets

    To create a minimal installation of Liferay so that only the essential Liferay portlets are available, uncomment the following block and set the include attribute to false for the portlets you want to remove. To make a portlet available, set the include attribute to true. The struts-path attribute is shown so that it's easier for the editor of this file to associate a portlet id with a portlet.
-->

<portlet>
          <portlet-name>33</portlet-name>
          <struts-path>blogs</struts-path>
          <include>true</include>
</portlet>

I then rerun Liferay portal using Eclipse. For my surprise I could find the Collaboration category in the Add Content window with the Blogs entry available:

Hope this shortens the path when you come to need to turn a portlet on/off.

On September 1st I completed 1 year at Chemtech.

So far I’ve worked on projects that demanded what I knew and what I didn’t know yet. This is the kind of thing I ever wanted. While I was looking for a job opportunity this was what I had in my mind.

The Computer Engineering graduation helped me to learn the basics so that I could be prepared to start working.

The real learning happens when you get to the market and start building real software that controls a huge industrial plant for example as is the case of a control panel that shows to the users the state of variables scattered across a huge industrial field consisting of a lot of machines and processes. Software that you build, test, homologate and deploy in production environment. Software that is built in partnership with the client - be it through e-mail, by phone or in person. This kind of thing gives you a valuable experience.

The projects I’ve worked on during this 1st year are the following:

Braskem - Control Panel
ASP.NET, C#, CSS, NHibernate, SQL, ORACLE, IIS

Braskem - Billing and Measurement System
ASP.NET, C#, CSS, NHibernate, Web service, SQL, ORACLE, IIS

Chemtech - Site (nicknamed chemsite)
Liferay, Java, CSS, SQL, MySQL, Tomcat

CSN - MES
VB 6, SQL, MS SQL Server

As you see I’ve used different technologies in each project. This strengthens your brain as you advance in the learning path and gives you the mindset you need to carry on so that you can work in the front line with more responsibilities and more exciting and engaging projects.

During 1 year I could learn a lot of things and made the difference constructing software products used by a lot of people (hey, take a look at the about me section at the top right side of the blog). See that I wrote a lot of people instead of millions of people. Someday I’ll achieve that "millions of people" :o)

Chemtech is a great company to work for. People are always ready and motivated to help you. Even my manager once in a while helps me. He sits by my side and starts coding with me! Everyone wins when one member of the group wins. This is the thought!

All in all, great company and great people together. That’s why Chemtech is making the difference in Brazil’s engineering scenario.

If you’re looking for an engineering job opportunity in Brazil, consider Chemtech. A company that thinks ahead of its time envisioning a great future for Brazil and the world.

I could enumerate lots of good points about Chemtech, but if you want to know more, go ahead and visit the chemsite. There you’ll find a bunch of information.

Despite a great start in my professional career I’ve passed through some life changing experiences as for example, moving from Volta Redonda to Rio de Janeiro. Wow, this moving has made me think about different aspects of life. Many things I thought before getting in Rio de Janeiro needed to be rethought. In the end you grow mentally, emotionally and spiritually. This is the time in life when everything you’ve learned has to be put in action.

All in all, I’m doing at Chemtech what I like to do and that’s all folks for my 1st year.

A big thanks to everyone who has helped me to get here. I won’t name anyone. I’ve known a lot of people. People that helped me a lot.  You know who you are.

Thanks Jesus for making real what I’ve asked you, for being with me all the time and for teaching, capacitating and advising me through this journey/endeavor.

During the second half of July and the first half of August I was working on Chemtech's site doing some maintenance. It was a good job because I could get to know new technology as is the case of Liferay. Liferay is a great enterprise portal that allows you to create a complete website solution coded in Java.

I also could verify the quality of the new Eclipse 3.5 IDE codenamed Galileo that I used during the maintenance. It's a great IDE to Java developers. It has lots of plugins that allow you to work with practically any kind of programming technology inside a fantastic set of windows for every type of task. Before using Eclipse I had only worked with NetBeans to do Java development.

I improved my skills about Tomcat too.

During a time like this you accelerate the learning process and get to know new things which are very important for any software developer.

After fixing some bugs on the site and writing in chemsite’s project wiki everything I grasped and did I came back to Volta Redonda for a two week job on CSN's MES; more on this in the next post.

You see, for the past 10 months I’ve worked with ASP.NET and Oracle (Braskem) and then I switched to work with Java and MySQL for 1 month (chemsite). Now at CSN I'm working with Visual Basic and SQL Server.

This shows that in today's world there's no bullet proof technology when we talk about programming languages and database systems. Each company has its own legacy systems that date back to two or one decade ago and such systems require certain types of interfaces developed in certain types of technologies. For example, if you look to 10 years ago (1999), C# wasn't even a programming language and so Visual Basic predominated during that time. It is most of the times impossible to a company to redevelop a really big system in a new programming language that is today's bullet proof. Those big systems consumed a lot of time and money to be constructed and in the future the programming language that is today’s bullet proof may not stand out.

As software professional you must act with any tool that is put in your hands.

I like what I do and no matter the tool I use I'm always satisfied with my job because of course, I do what I like to do, that is, software development.

To explain why Chemtech is a great place to work for in Brazil I think the above text says it all. In just 1 year I had the opportunity to work in different projects that use different technologies. A great way to leverage a career.

Thanks Jesus for that! :)

I extracted a lot of Ids from a database table and needed to pass such Ids as a parameter to a webservice method. The webservice method was expecting a parameter of type List<long>. I didn’t find a way of passing such a list to the webservice using the built in webservice form constructed by Visual Studio. The cause is that a List<long> isn’t a primitive type.

Talking with my peers I learned of a tool called soapUI. It’s a tool used to test webservices. Using it I could pass the list of Ids.

I created a new project in soapUI passing to it the webservice WSDL URL and I was ready to go.

 

This is the value I’ve put in Initial WSDL/WADL:

http://localhost:7777/WebServices/MyWebserviceName.asmx?WSDL

After clicking OK, soapUI will then load the webservice definition.

Clicking in Request 1 as shown in the following picture, the XML of a SOAP envelope appears so that we can test the webservice method.

The problem now was that I had a file called “input.txt” with only the Ids – each one in its proper line. The XML of the SOAP envelope expect that each id be passed in the format:

<ns:long>?</ns:long>

For example,

<ns:long>7</ns:long>

As we can observe, my input data don’t fit the pattern required by the XML.

To put my data in conformity with the XML I created a small but useful application called LinePrefixerSuffixer that receives the name of an input file containing the the initial data, the text to be “prefixed” in the start of each line, the text to be “suffixed” in the end of each line of the file and the name of the output file.

So for example, to comply with the above pattern, I’d call the console application with:

LinePrefixerSuffixer input.txt “<ns:long>” “</ns:long>” output.txt

Let’s say I have a file called input.txt with 1000 numbers in the same directory of the LinePrefixerSuffixer.exe executable.

Each line of the input.txt file has a number as:

1
2
3
4
5
6
7
.
.
.

Running the above command line in the command prompt I’d get a file called output.txt with each line now in the format I want, that is:

<ns:long>1</ns:long>
<ns:long>2</ns:long>
<ns:long>3</ns:long>
<ns:long>4</ns:long>
<ns:long>5</ns:long>
<ns:long>6</ns:long>
<ns:long>7</ns:long>
. 
. 
.

The C# code of the app is as follow:

using System;
using System.Collections.Generic;
using System.Linq;
using System.IO;

namespace LinePrefixerSuffixer
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                // Read in all lines of the file using query expression (LINQ).
                // I "prefix" the start of each line with the content of args[1] and
                // "suffix" the end of each line with the content of args[2].
                IEnumerable<string> fileLines = from line in File.ReadAllLines(args[0])
                                                select args[1] + line + args[2];

                // Writing the prefixed and suffixed file lines to a file named with the content of args[3].
                File.WriteAllLines(args[3], fileLines.ToArray());

                Console.WriteLine("Operation done.");
            }
            catch(Exception e)
            {
                Console.WriteLine("Use: LinePrefixerSuffixer <input.txt> prefix suffix <output.txt>");
            }
        }
    }
}

Now I can pass the content of the output.txt file to soapUI without worrying about having to manually prefix/suffix each line of my input.txt file:

Summary
In this post we saw how to build a simple but powerful application that prefixes and suffixes each line of a file.

We’ve used concepts related to file handling and LINQ and with only 4 lines of code we could manage to accomplish the task.

I think this shows how powerful modern programming languages as C# enables a clean and beautiful coding experience.

Hope this helps.

Visual Studio C# Console Application
You can get the Microsoft Visual Studio Project and the app executable at:

http://leniel.googlepages.com/LinePrefixerSuffixer.zip

Note: As this program uses LINQ, you must have Microsoft .NET Framework 3.5 runtime libraries installed on you computer. You can get it at:

http://www.microsoft.com/downloads/details.aspx?FamilyID=333325fd-ae52-4e35-b531-508d977d32a6&DisplayLang=en

References
[1] soapUI - the Web Services Testing tool. Available at <http://www.soapui.org>. Accessed on January 20, 2009.

[2] Sam Allen. File Handling - C#. Available at <http://dotnetperls.com/Content/File-Handling.aspx>. Accessed on January 20, 2009.

[3] LINQ. The LINQ Project. Available at <http://msdn.microsoft.com/en-us/netframework/aa904594.aspx>. Accessed on January 20, 2008.

[4] LINQ. Language Integrated Query. Available at <http://www.leniel.net/2008/01/linq-language-integrated-query.html>. Accessed on January 20, 2008.

Last Friday, January 16th, I was complimented by Chemtech. The company gave compliments to all employees that finished their college course in 2007/2008.

 
The computer engineers including me beside Rubião from right to left.

We got together at the 23rd floor of Rio de Janeiro’s office where Luiz Eduardo Ganem Rubião (CEO and founder of Chemtech) talked a little bit about the company’s perspectives giving us some insight related to life, economy, the job market and ongoing and future projects.

The way Rubião thinks about life subjects, mainly being an optimist is how I face the day to day. There’s no time to be wasted with illusions and we should always think positive. Doing so we’ll for sure harvest good fruits.

The following is the transcription of the letter I received with a present (a book of my area of specialization - computer engineering) …

Rio de Janeiro, January 16th, 2009

Dear Leniel Braz de Oliveira Macaferi,

Young people like you represent the future of technology. With your know how you can contribute to Brazil’s development so that it stands out in the worldwide technology. It is in the knowledge, in technology that a differential arises.

Chemtech is proud to have been with you in your first steps to engineering and for us still being together. It’s with satisfaction that today we call you an engineer! A Chemtecheano engineer, a Brazilian engineer.

We’re together in this journey that starts with your graduation. Success!

A strong hug from the Chemtech family.

It's with great pleasure that I'm writing this post. I finally got a job after 5 months of eager expectation. I'm going to work at Chemtech.

I've been sending lots of resumes to almost all IT related companies from Brazil and to some international companies since I got out of ITA-Petrobras.

Fortunately, on July 10 I received an e-mail from Chemtech asking for my grade transcripts related to the computer engineering course. I then sent the transcripts to them on July 30. On August 28 I was invited to participate in a in house interview that would take place the next day. On August 29 I went to Rio de Janeiro. During the interview conducted by two managers and one software developer, a job proposal for a Junior Analyst position was made and I accepted. The area of specialty is computer engineering.

I dreamed about getting a job in a top technology company. From the moment I began the Computer Engineering course in 2003 my constant thought has been to get a position in a company that is attractive.

Today what was a dream is now pure reality.

I'm extremely motivated to be a part of the Chemtech team!

I thank Jesus Christ for helping me to achieve this goal. After all, it's for Him that I live! :)