HTTP Compression with Apache Axis, a huge performance boost !

As mentioned in my previous post, I was researching/benchmarking on how HTTP Compression with Apache Axis improves performance. The results I got post HTTP compression, are surprisingly amazing i.e. “response time of web service calls is reduced to one third and one forth in many cases”.

Though HTTP Compression is a generic topic, not related to Salesforce, but for my benchmarking, I used Salesforce SOAP web services.

Good news : I got a highly noticeable response time difference with each Partner, Enterprise, Apex and Metadata web service calls.

How to make HTTP Compression work in Axis (Salesforce)

Prerequisite – setup Classpath with dependencies

Apart from WSDL2Java classes for your partner, metadata etc WSDL. You need following jars in classpath.

• axis.jar (I used Axis v1.4)

• commons-httpclient-3.1.jar 

• common-codec-x.jar (If you are getting this error http://bit.ly/9dBisn)

• Other jars that are usually included when working with Salesforce Webservice + Apache Axis 1.4.  Like jaxrpc.jar, commons-lang-x.x.jar, commons-discovery-x.x.jar, wsdl4j-1.x.x.jar. All these dependencies are available with your axis 1.4 binary download. Check the “axis-1_4.zip\lib” folder.

Step 1 – Extend Axis Service to @override createCall() method

As explained in this post by Simon. We need to extend the WSDL2Java generated extension class to “org.apache.axis.client.Service” as shown below. In case of Salesforce partner.wsdl, this class is SforceServiceLocator.

import javax.xml.rpc.Call;
import javax.xml.rpc.ServiceException;
import org.apache.axis.transport.http.HTTPConstants;
import com.sforce.soap.enterprise.SforceServiceLocator;

public class SforceService extends SforceServiceLocator {
   public Call createCall() throws ServiceException {
      Call call = super.createCall();
      call.setProperty(HTTPConstants.MC_ACCEPT_GZIP, Boolean.TRUE);
      call.setProperty(HTTPConstants.MC_GZIP_REQUEST, Boolean.TRUE);
      return call;
   }
}

If you are working with other Salesforce WSDLs, then you have extend these classes in the exact same way. 

• Apex WSDL : com.sforce.soap._2006._08.apex.ApexServiceLocator

• Enterprise WSDL : com.sforce.soap.enterprise.SforceServiceLocator

• Metadata WSDL : com.sforce.soap._2006._04.metadata.MetadataServiceLocator

Note: We just need to change “extends SforceServiceLocator” in above code snippet to these classes, rest of the code will remain the same.

Once you are done with the change then you client code needs, to use this extension class for creating the stubs, as shown below

SoapBindingStub stub = (SoapBindingStub) new SforceService().getSoap();
// prepare the stub by either making a login call 
// OR
// by setting headers for SessionId/ServerURL etc
QueryResult qr = stub.query("Select Id, FirstName, LastName from Contact");

// Similarly you can create stubs for Apex, Metadata and Enterprise WSDL 
// as shown below
// Apex WSDL
ApexBindingStub apexBinding = (ApexBindingStub) new SforceService().getApex();
// Metadata WSDL
MetadataBindingStub apexBinding = (MetadataBindingStub) new SforceService().getMetadata();
// Enterprise WSDL (looks same as Partner, but SoapbindingStub here is 
// under package com.sforce.soap.enterprise.SoapBindingStub
SoapBindingStub stub = (SoapBindingStub) new SforceService().getSoap();

Step 2 – Tell Axis to use “CommonsHTTPSender” instead of “HTTPSender”

One way to do that is create a config file named “client-config.wsdd”, as explained by Simon. Here is the relevant text from his blog post :

“you have to configure axis to use the vastly superior CommonsHTTPSender class rather than the default HTTPSender class as its transport, this is done by changing the transport entry in the client-config.wsdd file from java:org.apache.axis.transport.http.HTTPSender to java:org.apache.axis.transport.http.CommonsHTTPSender and adding Commons HTTP to the class path.”

You can Google around how to create a client-config.wsdd file for this. I usually hate creating config files, so here is an alternate approach if you want to achieve the same using following Java code (Add this code inline to some class that’s called on application startup or prior making SOAP calls)

// You need to include this static block
// in some startup class or just set the AxisProperties
// some where else.
static {
   AxisProperties.setProperty(
     EngineConfigurationFactory.SYSTEM_PROPERTY_NAME,
     CompressedEngineConfigurationFactory.class.getName());
}

public static class CompressedEngineConfigurationFactory implements
  EngineConfigurationFactory {

  public static EngineConfigurationFactory newFactory(Object param) {
    return new CompressedEngineConfigurationFactory();
  }

  public EngineConfiguration getClientEngineConfig() {
    BasicClientConfig cfg = new BasicClientConfig();
    cfg.deployTransport("http", new SimpleTargetedChain(
      new CommonsHTTPSender()));
    return cfg;
  }

  public EngineConfiguration getServerEngineConfig() {
    return null;
  }
}

That’s it guys, try making calls now. You have HTTP Compression “ON”.

Step 3 – Feeling the response time difference

To really feel, “how SOAP compression is great”, try benchmarking Salesforce web service calls that return huge response. For ex. a SOQL query that returns 2000+ records, or a heavy metadata call.

When not to use HTTP Compression ?

Http compression(GZIP) is good for web service calls working on “TEXT” request/response. If your web service call is returning something binary like ZIP files. I would suggest, don’t use HTTP compression on that, because that is already compressed.

So ideally, one needs to decide “When to turn on HTTP compression ?”, based on Web service Response Type.

Do “SFDC WSC API” needs HTTP Compression ?

What is SFDC WSC API ?

SFDC WSC API i.e. The Force.com Web Service Connector (WSC) is a high performing web service client stack implemented using a streaming parser. WSC also makes it much easier to use the Force.com API (Web Services/SOAP or Asynchronous/BULK API).

Http Compression with WSC !

Good news here is that, HTTP Compression is “ON” by default with WSC. So one needs to do no tweaks to get it with WSC. I must say, great job done by WSC team, the API is very easy to use and gives many hidden cool stuff !

If required, you can easily turn HTTP Compression “OFF” by calling

com.sforce.ws.ConnectorConfig.setCompression(false);

What do you say ?

Let me know, how Axis compression works for you. Would be interested in getting your feedback on this..!