MaintainJ Blog

September 27, 2009

How to trace socket servers or non-GUI apps?

Filed under: Uncategorized — maintainj @ 6:23 pm

All the MaintainJ video demos show analyzing GUI applications, but you can also analyze non-GUI applications (for that matter any non-J2ME Java code). One query I got was how to use MaintainJ on a socket server application. Socket server applications typically do not have a GUI and it is not easy to define the start and end points of a use case. This post addresses this issue. The same procedure can be applied to to trace a single method or GUI applications when you do not want to manually turn tracing on and off.

The key here is to identify the entry call to the application after which all the calls must be traced. Tracing will be turned on before this method is entered and turned off immediately after this method exits. The call trace is written to a file with name ‘traceFileXX_threadName.ser, where XX is the trace file count starting from 1.

For a Servlet based application, the entry call could be MyAbstractServlet.service(..) method, where all the Servlets in the application override or implement service(..) method. Typically this will be a ‘facade’ or ‘entry’ method in an abstract class or interface where all processing starts with a call to this method.

Once such a method is identified, you need to manually change the aspect (aop.xml) file as follows:

<aspectj>
<aspects>
<concrete-aspect name=”com.maintainj.inst.MyAspect” extends=“com.maintainj.aspect.HeadlessExecutionAspect”>
<pointcut name=”anyExecution” expression=”execution(* *.*(..)) || execution(*.new(..))”/>
<pointcut name=”entryCall” expression=”execution( * com.xxx.MyAbstractServlet.service(..))”/>
</concrete-aspect>
</aspects>
<weaver options=”-proceedOnError -nowarn”>
<include within=”com.xxx..*”/>
<exclude within=”com.maintainj..*”/>
</weaver>
</aspectj>

Pay attention to the changes in blue. The first change is the super aspect that is extended, which is HeadlessExecutionAspect. This is same for any type of application, Java/J2EE/Eclipse plug-in, as long as you want to generate the trace files in headless mode. If you are capturing the method call context, change HeadlessExecutionAspect to HeadlessExecutionDataAspect.

The second change is the pointcut. If in the auto generated aop.xml there is a second pointcut below ‘anyExecution’ pointcut, delete that and add the new ‘entryCall’ pointcut. This is where you are defining the entry call to your application. The class name here can either be an abstract class or interface or a concrete class. Ensure that the ‘facade’ class is included in the instrumentation scope.

You can start with the auto generated aop.xml and then change it. The location of the auto generated aop.xml for various wizards is discussed in this FAQ. If you are using one of the MaintainJ Run Configurations, in the ‘Aspect’ tab check the ‘Do not regenerate aop.xml’ so that your changes will not be overwritten.

The trace files are written to the current folder from where the application is started. Locate the trace files and copy them to a project in Eclipse workspace to view the beautiful MaintainJ diagrams!

Choudary Kothapalli.

September 17, 2009

The premise of MaintainJ and its road map

Filed under: Uncategorized — maintainj @ 3:55 am

This a long post and is divided into the following sections.

1. Java applications are very hard to read and understand
2. That’s what we have debuggers for, right? Wrong.
3. The key reasons for complexity in understanding Java code
4. Does it have to be this difficult to understand and maintain code?
5. How does MaintainJ help?
6. MaintainJ roadmap

1. Java applications are very hard to read and understand.

Let’s say we (Java developers) need to trace what happens on press of a button in a web page of a J2EE application. To keep the discussion simple, let’s assume that the application is a simple Servlet based web application. If we were to trace by reading the code, this is how it would go.

1. First we will read the HTML to figure out the servlet url called.

2. In the web.xml, we will look for the ‘servlet-mapping’ that matches with the request url. Using that information, we can figure out the actual servlet class called.

3. There may be some filters defined in web.xml that are invoked before the servlet. We need to figure them out by reading the ‘filter-mapping’ nodes. These nodes match in two ways – by ‘url-pattern’ or ‘servlet-name’. We need to find the correct ones.

4. Using the filter names figured out in the last step, we can find the actual filter classes called.

5. Each filter class may be forwarding to another filter or to a servlet or JSP. This decision will be made based on the runtime data in the request. We wouldn’t know that by code reading.

6. There may be multiple filters matching the same request. The sequence in which they are called needs to be figured out. The sequence depends on how the filters are mapped – whether by ‘url-pattern’ or ‘servlet-name’.

7. The filters may be invoked or not depending on the value of ‘dispatcher’ node. The possible values are request/forward/include/error. Depending on how the current request is dispatched and the value of dispatcher node, the filter will either be invoked or ignored.

8. There may be some servlet listeners defined. These listeners listen to the request, session and context events. They can change the servlet request, session or context. We need to check what these listener classes are doing.

9. The listener node does not mention the type of the listener. We need to open the class and check its super class to figure out whether it is listening to request or session or context events.

10. Most probably there will be some ‘init-param’ nodes defined for the servlet. The servlet behavior is determined based on the values of these parameters.

We can easily flog this to death. The request didn’t reach the processing stage yet. It has to get validated, reach the business tier through all the controllers, facades, delegates, ejbs etc., may possibly update the database, make an asynchronous messaging call and come back all the way. There will certainly be more frameworks on its way.

The point is, there are so many possible objects and methods that may be called for any one user action. Only a few of them will actually be called. Finding the actual ones called, so that we can track down a particular behavior, is very difficult from code reading.

2. That’s what we have debuggers for, right? Wrong.

Let’s try using the debugger to understand the flow. First problem – where to put the break points? Take the case of listeners and filters. Are we going to put break points in all of them to figure out the ones that are called? Second problem – the control does not flow linearly. Say the control is currently in a listener. As we step through, it will go out of the listener class into the servlet API and then possibly enter another listener class. Same is the case with filters. We are not going linearly from component A to component B. The framework code comes in between and it makes debugging difficult. Debugging this way is tiresome and we can easily get lost after a while.

Debuggers help to check the behavior in a certain method or a component, but not to trace across many components.

3. The key reasons  for complexity in understanding Java code

The key reasons for this apparent complexity of Java applications are as follows.

a) Many things are known only at runtime. Code that uses interfaces or abstract classes or listeners are just a few examples. Neither manual nor automated code analysis would reveal much about what exactly is happening at runtime.

b) There is lot of framework or container code. The code we write, component code, is often glued together by the container code. It is difficult to understand how control flows from component A to component B through the container code. Then there is lot of configuration associated with these frameworks. Servlet, EJB, Struts, Spring are just few examples of these frameworks.

c) Java applications have to integrate with external systems like web, database, messaging, etc. Source code analysis does not tell how control flows into and out of Java code to the external systems. For example, there is no automated way to tell which database tables are used by a DAO.

All the above reasons make it very difficult to understand code by reading and debuggers don’t help much.

4. Does it have to be this difficult to understand and maintain code?

Most of the time we will need to change parts of our code to fix or add a feature. Regardless of all those frameworks we use, most of the business logic is in our code. So, if we can quickly get to that part of the code, our task will be a little easier. This is not to gloss over the importance of understanding how the frameworks work and behave. The reason for a particular behavior may be in the configuration file. But more often it is something in our code. So whether we are using Struts, Spring, EJB or whatever framework, if we could track our components that are called for an action, it is easier to understand and maintain the code.

Going back to the example we started with, most of our application logic resides in one of those servlets, filters or listeners. This will be true for any Java application using any number of frameworks. If we could quickly identify those components at play for a user action, we could quickly reach the root of the problem.

This is the premise of MaintainJ.

5. How does MaintainJ help?

MaintainJ captures the call trace at runtime for one user action. It captures the actual objects and methods called for an action. Then, it uses that information to render neat sequence and class diagrams. The sequence diagrams can be explored and are easy to read. One can easily understand what exactly is going on for an action and find the areas of interest quickly.

6. MaintainJ roadmap

Maintenance does not end with understanding the problem. But understanding is the first and important step. Next, we do change impact analysis, do the actual change and then regression test.  Here is a broad outline of how MaintainJ plans to add those features to the product.

a) Context information will be added to diagrams. This will be done in the next major release, V3.0. At every method call, the parameters to the call and the return value will be shown. All SQL calls going out to the database will be captured and shown. All request parameters coming into the web application will be shown.

b) Better searching capability will be added to the diagrams. Searching for classes or methods involved in an action (represented by a call trace file) will be added. This would help to find all the actions that will be affected by changing a part of the application. The ability to search by context (parameter of a method, say) will be added too.

c) Impact Analysis capability: The idea is, if we have all possible actions in the application recorded, we can tell which actions are affected by the changes to a set of methods or classes. As MaintainJ has information of all the runtime methods called for an action, the impact of a change in a method or class can accurately be mapped to a set of use cases. By knowing all the database tables (or views or sequences) used by a user action, we can deduce the actions impacted by a change to the database structure.

d) Better filtering techniques will be added to the diagrams. The existing filtering techniques – using instrumentation scope, removing all loop and recursive calls and the different ways to delete unnecessary calls – help, but are not sufficient. Sequence diagrams can get messy very easily. Better filtering techniques for the diagrams like filtering all log calls, get/set methods, tearing off a call into a new diagram, will be added.

e) Memory profiling will be added. MaintainJ didn’t start as a profiling tool. But as it is a runtime analysis tool, it is easy to capture and show the runtime statistics. Currently it shows the method call response times. Useful memory information will be added to this.

f) Regression testing capability: This feature will be added in version 4. Using the run time context data for each recorded action, test cases will be generated and used for regression testing.

In summary, context information and search capabilities in the sequence diagrams will add impact analysis feature to the product. Regression testing capability will complete the maintenance cycle.

September 16, 2009

Tips when using MaintainJ on WebSphere

Filed under: WebSphere — maintainj @ 6:59 am

Tip 1: For all other types of servers except WebSphere, a separate starter script is generated to start the server with MaintainJ. For Java applications, there is a separate run configuration. This will give the user the freedom to run the server/application with or without MaintainJ. As the applications take more time and resources when starting with MaintainJ, there will be times when one would like to start without MaintainJ.

But in the case of WebSphere, the server classpath and JVM arguments are changed in WebSphere admin console and the server is started the same way as before. Here is the tip to start WebSphere without MaintainJ.

The changes to server JVM classpath and arguments are stored in runtimes\base_v61\profiles\AppSrv01\config\cells\mainNode03Cell\nodes\mainNode03\servers\server1\server.xml . The path may be a little different on your PC depending on your server and node names. You can take a backup of this file before changing the settings for MaintainJ. When you want to start the server without MaintainJ, backup the server.xml with MaintainJ settings and use the original version.

Tip 2: This tip is more for RAD 6 users. RAD 6 seems to use different IBM JRE versions for different minor versions of WebSphere. We did the necessary fixes to make MaintainJ work for the reported problems, but in case the server fails to start after changing JVM classpath and arguments, follow the procedure below:

The changes to the server JVM classpath and arguments are placed in ‘processDefinitions’ element in server.xml. The path to this file is given in the last tip. You will find the MaintainJ related classpath changes in ‘classpath’ element. Remove them. The JVM argument related changes are in ‘jvmEntries’ element’s ‘genericJvmArguments’ attribute. Remove them as well. WebSphere will start normally after this. Contact us with your JVM version (with exact output of ‘java -ver’ command) if you face this issue.

Choudary Kothapalli.

September 15, 2009

What I mean by J2SE application type

Filed under: Uncategorized — maintainj @ 9:41 am

MaintainJ provides many wizards to automate instrumenting different types of applications. I often ask people the type of their application. I recently realized that the J2SE type as I mean it was confusing. The J2SE Run Configuration can really be used on any application that is launched in Eclipse using ‘public static void main(String[])’ method. It works even for J2EE servers if they are configured to be started by calling the main method on a server starting class.

One MaintainJ customer configured their JBoss server to be launched this way by calling the org.jboss.Main class. They have their application code setup as projects in Eclipse. When they use the ‘Java Application with MaintainJ’ to launch their server, it just works fine as described in J2SE Run Configuration.

 All these wizards just do the things described here, which is to generate the aspect and setup the classpath and JVM arguments for instrumentation. As J2SE and J2EE applications are typically started in different ways, different wizards are provided. But they do the same task at the end of the day. If your application doesn’t seem to match any configuration for which wizards are provided, please spend some time in understanding how instrumentation works as described here . Then you should be able to instrument your application easily.

MaintainJ can generate diagrams for any Java code except J2ME which AspectJ does not support. MaintainJ depends on AspectJ for instrumentation. So, even if your application server is not supported or if your application configuration does not seem to be supported, it does not mean that MaintainJ wouldn’t work for you. You just need to customize it a little to work for you. The wizards offer you a starting point.

Choudary Kothapalli.

Including calls from JSPs in sequence diagrams

Filed under: Uncategorized — maintainj @ 9:08 am

One question that pops up often is why JSPs are not shown in MaintainJ generated diagrams. The answer is, they will be shown if their package is included to the instrumentation scope. Below is a sample diagram generated on Weblogic server.

If MaintainJ.war is used to define the instrumentation scope, this can be done \in ‘Step 2: Configure Instrumentation Scope’. Or else, the aop.xml can be edited directly. But the packages where the servlet class files, into which JSPs are eventually converted, are put under different packages for different application servers. Below are the package names for the 4 popular servers:

Tomcat and JBoss: org.apache.jsp
WebSphere: com.ibm._jsp
WebLogic: jsp_servlet

Once this package is added to the instrumentation scope, the calls going out from JSPs will be shown in sequence diagram.

Choudary Kothapalli.

Powered by WordPress