Friday, July 19, 2013

How to send Basic Auth information when invoking a web service with SoapUI

I recently wanted to invoke a web service via SoapUI, which was expecting basic auth credentials. There are two options.

1. When you open a request generated by SoapUI, at the bottom you will find two tabs named Auth and Headers. When you click on the Headers tab, it will open a small window which allows you to add any headers to the message. There you can add a header name "Authorization" and its values should be "Basic <Base64 encoded username:password>".



If your username and password are admin:admin, then when you encode it, your header values will look like "Basic YWRtaW46YWRtaW4="

After setting this, if you send a request in SoapUI, you will be able to see the Authorization header in the request.

2. Other option is to use the Auth tab and define the username and password there. But, these information will not get sent as a header. To send them as a header, you need to do the following.

Open the "Preferences" window by clicking the icon or from the SoapUI menu. Then select "HTTP settings". From that settings list check (tick) the "Authenticate Preemptively" option as shown in the image below.



Then, set the username and password in the Auth tab as I described previously and send a request. You will see the Authorization header sent with the request as shown below.


I learned this from Charitha Kankanmge of WSO2 and thought of writing it down for others benefit.

Thursday, July 18, 2013

PublishWSDL option in WSO2 ESB explained...

If you are wondering what is the PublishWSDL option of WSO2 ESB, you can have a look at how to create proxy services in WSO2 ESB.

Lets say you are going to expose a web service by creating a proxy service. If you do not publish the WSDL of the web service you are exposing, then if you take the WSDL of the created proxy, it will only show the "mediate" operation. If a client wants to invoke the backend service via the proxy service, it needs to know the expected message format by the web service (parameters, their names etc.).

But, if you decide to publish the WSDL of the web service, then the WSDL of the proxy service will show all the operations which are available in the web service. This makes life easier to the client.
But publishing WSDL also exposes all the operations of the web service. What if you want to expose only a few operation to the clients? In that case, you can publish a WSDL with only the operations which you want to expose.

There is another advantage. Lets say the backend service operation expects three parameters to be sent (lets assume these are name,department and a permission level). We want the client to send only the name and department and we are planning to add the permission level to the request within the proxy service. In such a scenario, we can publish a modified WSDL which is expecting only the name and the department and inject the permission level within the proxy service.

Above are the use cases of PublishWSDL option available in WSO2 ESB. Although this is a simple thing, I couldn't find a place which explains this. So, hope this helps somebody.

Thanks Kasun Indrasiri of WSO2 for explaining this to me... :)

Saturday, September 29, 2012

Unix timestamp vs Java timestamp

A recent incident which occurred to me and a colleague of mine made me write this little post. We were setting a property in a hive script and was trying to retrieve some data comparing with the value we set. In other words, this is what we did.
  1. We set a property named max_ts to hive context within a Java class. We used the Timestamp.getTime() method to get the time.
  2. In the hive script, we were doing something like:
    1. Select my_columns from my_table where unix_timestamp(a_string) > ${hiveconf:max_ts}
 But this was not seem to be working. Even when we had records which has a timestamp greater than the max_ts, query was not returning them. Then we added some debug logs and realized what was going wrong.

Unix timestamp gives the number of seconds which has passed since the epoch.
Java timestamp gives the number of milliseconds since the epoch.

Although both of these uses the same epoch (midnight of January 1st 1970), they provide two different values. Thats why the query was not behaving as intended. Although I had read/learned about this difference, it took some time to kick it into the mind.
  

Saturday, September 15, 2012

Now you can send the SoapFaults to the Fault Sequence in WSO2 ESB

WSO2 ESB 4.5.0 was released recently. There are many new features, enhancements and performance gains in this new release. I wanted to highlight one new cool feature which I couldn't wait anymore. That is the title of this post.

In the previous releases of WSO2 ESB fault sequence was hit only when there was a fault during the mediation process taking place inside the WSO2 ESB. If a soap fault was received from a backend service it was sent to the out sequence because the WSO2 ESB considered it as a response from the backend. It was fair from ESB's point of view and unfair from developer's point of view. But with the 4.5.0 release, now you can send the soap faults to the fault sequence and do the fault handling.

To do that, you need to set the FORCE_ERROR_ON_SOAP_FAULT property before you send the message to the backend. This is a newly introduced property and following sample proxy service will show you how to do it.


   
      
         
         
      
      
         
      
      
         
            
         
      
      
         

Hope this property will help many integration developers during their use of WSO2 ESB.

NOTE

Above proxy configuration was taken from a resource which was emailed to me by Kasun Indrasiri who is an ESB expert in WSO2.

Happy Integration.... :)

Sunday, May 20, 2012

Getting ActiveMQ Redelivery to work with WSO2 ESB

Recently I wanted to use the redelivery mechanism available in Apache ActiveMQ while using it with WSO2 ESB. My scenario was like this.
  1. Retrieve a message from a JMS queue
  2. Write the message to a file URI (in your case you may want to send it to a given endpoint)
  3. If writing to the file is not successful due to some failure, I wanted the message to be in the queue and ActiveMQ to try repeatedly writing to the file.
Following are the properties/parameters I had to put in the axis2.xml of WSO2 ESB (under JMS transport receiver). Remember that you need to put the other parameters such as initial context factory, provider url, connection factory and connection factory type etc.
<parameter name="transport.Transactionality" locked="true">jta</parameter>
<parameter name="transport.jms.SessionTransacted" locked="true">true</parameter>
<parameter name="redeliveryPolicy.maximumRedeliveries" locked="true">-1</parameter>
<parameter name="redeliveryPolicy.redeliveryDelay" locked="true">2000</parameter>
<parameter name="transport.jms.CacheLevel" locked="true">consumer</parameter>

I will explain some of the above parameters.

  1. transport.Transactionality - This is the desired mode of transactionality. I was using distributed transactions. Therefore the value was jta. If you are using local transactions you can put it as local.
  2. transport.jms.SessionTransacted - Whether the JMS session should be transacted or not.
  3. redeliveryPolicy.maximumRedeliveries - Maximum number of retries you wish. If set to -1, ActiveMQ will infinitely retry.
  4. redeliveryPolicy.redeliveryDelay - Delay between retries. I have set it to 2 seconds (i.e. 2000 milliseconds).
  5. transport.jms.CacheLevel - This is the most important property to get the redelivery mechanism to work properly. This has to be set to "consumer". Reason for this is ActiveMQ  RedeliveryPolicy is dictated by CONSUMER, not PRODUCER.
With above configurations, you are able to get the redelivery mechanism in ActiveMQ to work when used with WSO2 ESB.

You will also need to set the SET_ROLLBACK_ONLY property in the fault sequence of your proxy service. Otherwise the transaction will not get rollbacked.
<property name="SET_ROLLBACK_ONLY" value="true" scope="axis2"/> 


Following links will provide further help.
1. WSO2 ESB documentation's JMS transport section
2. ActiveMQ redeliverPolicy configuration parameters

Thursday, September 1, 2011

Fronting your Stratos service with the WSO2 LoadBalancer

Today I wanted to front the StratosLive Manager with the WSO2 LoadBalancer. I didn't have an idea on how to do it. Then with Afkham Azeez's instructions, I was able to do it. I thought of recording the steps for my and others future use. Following instructions will guide you to front any stratos service with a loadbalancer.

First you need to have the WSO2 LoadBalancer and the service (in this case WSO2 Stratos Manager).

Changes to be done in LoadBalancer
Find the loadbalancer.xml in the repository/conf folder.
Add the following lines under services tag.



cloud-test.wso2.com

wso2.manager.domain



This host name is important because it is going to be added to the axis2.xml of Manager.

Add the above host name to /etc/hosts file.

Changes to axis2.xml of Manager
First enable clustering .



Then set membership scheme to well known addressing - wka


wka

Then set the domain as wso2.manager.domain. This is the host name added in the loadbalancer.xml.


wso2.manager.domain

Change the local member port to some random value, say 5000.


5000


After setting the above parameters, start the LoadBalancer. Then start the Manager. When the manager joins the cluster, you will see a log similar to the following in the LoadBalancer log.


[2011-09-01 23:55:55,919] INFO - RpcMembershipRequestHandler Received JOIN message from 192.168.1.2:5000(wso2.manager.domain)
[2011-09-01 23:55:55,921] INFO - MembershipManager Application member 192.168.1.2:5000(wso2.manager.domain) joined group wso2.manager.domain
[2011-09-01 23:56:06,935] INFO - DefaultGroupManagementAgent Application member Host:192.168.1.2, Port: 5000, HTTP:9763, HTTPS:9443, ACTIVE:true joined application cluster


Now you can try to access Manager's management console via the following url https://cloud-test.wso2.com:8243/

Note that Manager is running on 9443. But we specify the port as 8243 because it is the port LoadBalancer is running. Then, the above request first goes to the LoadBalancer and then to the Manager. When you enter the above url in the browser, you can see this by monitoring the two logs of LoadBalancer and Manager.

NOTE: This is the most simplest scenario fronting a service with the load balancer.

Thursday, August 25, 2011

Billing in cloud computing - StratosLive billing explained

StratosLive comes with its own billing system to bill the tenants for using the cloud. Lets have a look at the bill generation process of StratosLive. Shown below is a simple diagram to explain the base entities involved in bill generation.

There can be multiple billing engines in your environment. In StratosLive there are two of them. One is for scheduled bill generation (monthly invoice) and the other is for on-demand bill generation (interim invoice). Both these engines use a single billing database. Each of these engines have a set of handlers to be executed. These handlers can be configured via the billing-config.xml. These handlers are executed by the engine at the time of bill generation. Shown below is segment from the billing-config.xml










multitenancy-billing-rules.drl






email-billing-notifications.xml





What are the data used in bill generation?

  • Subscription details – subscriptions of the user for the duration considered in the bill generation. There is a subscription entry for each usage plan which the user was subscribed to during the billing period
  • Usage details – Usage details of the user for the month of bill generation. This includes bandwidth usage (webapp bandwidth, service bandwidth and registry bandwidth) and registry storage of the user.
  • Pricing strategy – This defines how users are going to be charged for subscription and overusage. Shown below is a sample xml file which defines the pricing strategy.



10

10
0


25
0.1


2000
0.1




When does bill generation happen?

As I mentioned previously, monthly invoice is generated by a scheduled task. This is achieved via Synapse scheduled tasks. When the billing manager is started (i.e. when the server starts), bill generation task is scheduled for the billing engine. This task scheduling can be configured via the billing-config.xml.

Lets go through the bill generation step-by-step. Billing engine for scheduled bill generation executes the following set of handlers.

1.SubscriptionFeedingHandler – Retrieves the subscription entries relevant to the bill generation period. For each subscription, relevant user's usage data also retrieved. Subscription entries are added to the BillingEngineContext to be used by other handlers.

2.SubscriptionTreeBuildingHandler – Goes through all the subscription in the context and arrange the subscriptions of same customer together. Creates invoices and assigns the subscriptions to them. Fills some data of the invoice such as invoice dates, payments falling under the billing period.

3.RuleHandler – Rules are used to calculate the subscription fees. Rule handler executes the rules over the billing engine context. This will assign a subscription fee for all the subscriptions in the contex.

4.InvoiceCalculationHandler – Over usage charges are calculated by this handler according to the pricing strategy define in a xml file. Total cost, total payments an carried forward values are also calculated. At this moment, there is a complete invoice for all the customers in the memory.

5.Finalizing Handler – This handler takes care of storing the invoices in the billing database.

6.EmailSendingHandler - After generating and storing the invoices, customers have to be notified. It is done by the email sending handler. When this is done you receive an email notifying you about the invoice.

Diagram below gives an idea on what happens in the first four handlers.




Scheduled vs. On-demand bill generation

These two operations are carried out by two billing engines

Scheduled generation executes two additional handlers than the on-demand generation. They are the finalizing handler and the email sending handler. When the invoice is generated on-demand, it is not stored in the database (because it is only for presenting to the user). No email notification is necessary because the invoice shown in the UI. That is the reason for FinalizingHandler and EmailSendingHandler not needed for on-demand bill generation.

Flexibility

The whole bill generation process is execution of set of handlers. If you want to add something new to the flow, you can simply write a handler. Then you need to mention that handler in the billing-config.xml for it to be executed by the engine.

Billing component will be further improved to calculate all the costs via rules. Then it will be super easy to make changes in cost items and pricing because only what we need is to edit the rule file. This avoids any code level change when it comes to changes in calculating the cost which is a huge advantage.