Thursday, July 19, 2012

Eclipse, never again!

For a long time I have been using Intellij IDEA and my last experience proved how accurate my decision was.


I had to use Eclipse due to obligations in a company. The feature I had developed was tested well and it was ready to be tested on server.


However; I had to spent a day to see it on server for a feature I had spent one day to develop. Problem was a "class not found" exception which I could not solve although debug it until to native binding of class loaders. The class was in related jar under web-inf lib folder; what was the problem? Perhaps it was an init / static block issue; but there were no clue...


After spending a day; I noticed that although the project I was working was a maven project and had been imported to eclipse via maven plugin; test folders were being included to build path when you execute publish/deploy on application server. Yes, for eclipse there is not such a concept called "test folders". 


I was so happy to find it out; after spending a day on this meaningless error. Then; i recognized how nice IDEA organizes it; execute coverage reports; analysis etc. And you do not have to install a maven plug-in or any other svn/git etc. plugin, everything is already added for you and they work. 


IDEA is really intelligent and I enjoy to work with it. I remember my first experience with IDEA. I had preferred Eclipse at the end of day; but as people keep suggesting it again and again; I gave it a chance again and now what I think is IDEA is the best java ide.


Other products of Intellij are great too:  RubyMine for ruby/ror, Pycharm for python ide. There are not real opponents in these areas as far as I know.


Okay Eclipse is open source and free but Intellij really deserves what it asks for; and if you are a developer; it's not a big amount to think: It's great tool to do your job efficiently.











Tuesday, July 10, 2012

Apache Synapse, WSO2; keep them in you mind!

After working with IBM Datapower; I googled around to see if there was an open source mediation alternative. I had thought I found a topic which was not covered by open source projects and I would start it immediately.But I was wrong; there was already a nice one by Apache. Although Synapse is not an %100 alternative to Datapower, it is a nice mediation project.

Immediatelly I thought I still had chance to improve it with config editors, management, monitoring etc. Again it was already implemented by another project: wso2.

Wso2 includes Synapse and add features on top of it. I recommend you to check these projects if you have not already. Just keep them in your mind that such projects exists; you may need them in future. 

Now I am bit regret to implement my previous mediation project with Spring Integration. I could make use of these.

Tuesday, June 26, 2012

AggregatedXmlMessageValidationException and inbound-gateway

I wrote a mediation layer with spring integration for a system. Mediation layer validates, transform, signs and encrypts given xml and then route it to required target.

However client is right to complain about the failure message for an xml which does not validate against the xsd schema. I am using validating-filter with spring integration and ws client recieves following message on xsd validation failure:




In order to fix that add an error channel to ws inbound such:



And finally the logger; it logs the detail of the problem to log and also returns it to client; so he/she can fix her code for given problem:



Tuesday, June 12, 2012

Custom XSLT Factory within application scope

Yesterday I tried an xsl transformation with the following xsl:




That xsl helps me to transform an xml to an xml which have scoped default namespaces. However that xsl is not working with Xalan. It is ok with Saxon. So I passed

-Djavax.xml.transform.TransformerFactory=net.sf.saxon.TransformerFactoryImpl to jvm.

It is okay with Tomcat but when you deploy to IBM Websphere application server; you recieve:

"[6/12/12 10:09:57:586 EEST] 0000000d UserRegistryC E SECJ0281E: Error creating user registry object. The exception is javax.xml.transform.TransformerFactoryConfigurationError: Provider net.sf.saxon.TransformerFactoryImpl not found" error.

The reason is that saxon.jar is only on application path; but we ask server to use it everywhere; so it looks for that factory as soon as server is running. One solution is to put that jar in the shared folder of ibm webshepre; so that it can locate it.

But then I think not to do it; let ibm webshere (and other deployed applications) to use it own xslt factory and my application use it's own xslt factory.

I was using Spring Integration. Spring Integration has a nice transformer (XsltPayloadTransformer) class; but it does not let you specify the xslt factory; it uses system variable.  So I extend it as:



I was lazy to extend XsltPayloadTransformerParser as well; so in my spring configuration file; I defined it as below:

Thursday, May 3, 2012

Spring Integration: WS Endpoints with Encrypting&Signing WS Services and testing with SoapUI

It took some time and check various resources for me to setup an ws-endpoint with signing and encryption in Spring integration to setup. I spent most of my time with misspelled passwords, incosistent libraries, configuing soap ui, learning policy files etc.

In this post I am aiming you to show these in one shot.

First let me share you where you can download or clone source-fileshttps://github.com/yamanyar/SI-with-Secure-WS-Example. After clonning or downloading the sources, you can open the pom.file (i prefer idea rather then eclipse) and try/examine yourself.

What I have done in the project is; I configured a spring integration; there are three web service endpoints using same bean (in order to perform service) with different configurations:

  1. Regular WS
  2. WS authenticated with Usename/Password
  3. WS authenticated with Signature and requires encrption
Responses from server to client are not authenticated or encrypted; however you will get the idea from this example project and you will be able to add if need.

Let me give some brief information about following topics: Key stores and certificates, policy files, endpoints and soapUI settings for testing "WS authenticated with Signature and requires encrption" endpoint.


Key Stores and Certificates

Keystores are files which hold public-private keys or only public keys.

Private key's are only known by the owner and public keys are distrubuted clients.

Your clients (not the client applications you wrote; people/software demanding from your server) can use your public key to verify that it is really signed by you. If a document is signed with a private key, it can only be verified with matching public key. Since the matching private key is only known by the sender; you can be sure that that document is signed by him/her. In this case you need to obtain public key of him/her in a secure way. This is an another story; let's assume you know him and he send it via e-mail.

Your clients can also use your public key to encrypt a document and send to you. If a document is encrypted with a public key, it can only be decrypted with matching private key. Since you are only one who  knows that the private key (you hope so :) ), you can decrypt the document sent. 

In keystore files; you store your private-public key pairs and public keys (certificates of other trusted clients).

I believe this short introduction was enough for you to understand purpose of following commands, however you can check here to go deeper into these terms.

Let me describe each command above. Although they are on windows; you can run them on linux/mac as well,  it is distrubuted with jdk.

  1. In first command; we create a keystore. Password of keystore is test1234. This keystore is for client. Client owns this file and does not share with anyone :) In practice client must generate this.  Since information is very secret; we also encrypt private-public key pair with another password "client1pass". A keystore may have many enteries with different password. This entry is named (alias) "client1". (Note keystore file is generated automatically since it does not exist before).
  2. In the second command we extract public key from client private-public key pair as a certificate. Please note that we DO NOT extract private key. Although there are some ways to achieve it, keytool does not allow you to extratc it.
  3. Now server must trust the public key we had extracted on step 2. In order to achieve that we add the certificate we extracted at step 2 to server's keystore. (again this file is created since it does not exists before.). Server will use this public key to check signature validity in future. 
  4. Let's create a public-private key for server. So that client can encrypt a document using server's public key and server can decrypt it with it's private key.
  5. This time we extract public key of server as a certificate
  6. And we add server's certificate to client's key store.

Meanwhile let me tell you what truststore is. Truststore is a keystore too; where it is expected to put the public key's (certificates) of other parties (whom you trust) in it. Although I use certificate and public-key as it refers to same thing; it is not. Certificate contains public-key and muck more data which is not our topic now.

Great; we have keys. Lets's continue.

Policy Files


WS Policy files are configuration files; that describes the security policy of web services.

Following files are at server side. The first one mandates that client should send username&password while the second one asks user to encrypt soap body (with server's public key) and sign it (with client's private key).

You need to add response policies if you need to sign and encrypt ws responses. In that case; for responses; you could use private key of server to sign the document and public key of client to encrypt it.

Username and Password Only Policy:


Signing and Encrption Policy:

Let's skip details and continue with endpoints:

Endpoints

There are three endpoints with different intercaptors. I am not going to add all the source code here. Please clone the project to examine /META-INF/spring/integration/ws-endpoints-config.xml file. Let me summarize that configuration file:

URLInterceptorsService Bean
echologgerws-echo-inbound
secureEchologger,authenticatorWithDigestUsernameInterceptorws-echo-inbound
certifiedUserEchologger,authenticatorWithTrustedCertificateInterceptorws-echo-inbound

Difference between interceptors:
(note: security interceptors and dependant beans are defined in /META-INF/spring/integration/ws-security-config.xml)

authenticatorWithDigestUsernameInterceptor:
  • uses first policy file (username/password)
  • uses passwordValidateHandler as a callback handler

authenticatorWithTrustedCertificateInterceptor
  • uses second polic file (requtest sign/encryption)
  • uses a list of callbackHandlers: keyStoreHandler & springSecurityCertificateHandler


Why does authenticatorWithTrustedCertificateInterceptor uses two callback handlers?:

  1. First handler will check whether the certificate is in the private keyStore. If it is, it is valid. If the certificate is not in the "private keystore", the handler will check whether the current date and time are within the validity period given in the certificate. If they are not, the certificate is invalid; if it is, it will continue with the final step: Checking it with trust store. In our case; we use same keystore file as a keystore and truststore. However we still need to define both.
  2. With the help of second bean; we authenticate the user: Yes user has a valid certificate, but does he allowed to continue? Which roles will she/he have?
I added comments all around in the project. I hope you understand what I described until here. Now it is time to to setup soapUI.

With maven or your prefred ide; deploy this application to tomcat or jetty on your local under context "/hi". URL mappings are bound to hostnames. (localhost:8080/hi/service/echo and 127.0.0.1:8080/hi/service/echo is not same).


SoapUI Settings:

With SoapUI we will test first and third endpoints; I leave second one as an exercise :).

As you create a new project you are asked to select a wsdl file / url. Show the wsdl file in /wsdls/com/yamanyar/esb/services/XmlEcho.wsdl. It will create a soapUI project. 

Open the Request1 of XmlExhoSoap11Binding node and modify the endpoint such:
"http://localhost:8080/hi/service/echo". Then just play run button; message you sent will be replied back:




Now let's test third endpoint. Add another endpoint like:
"http://localhost:8080/hi/service/certifiedUserEcho"

If you run again, due to "request signature and encryption" policies you will recieve an error. You need to configure soapUI to sign and encrypt body of the soap message.

Double click on the project, click on "WS-Security Configurations" and then "Keystores". As seen on the image below we add a keystore (/policies/store/local/client1ks.jks (which contains private-public key of client and public key of server as described above, password was: test1234)):



Again, as seen in the image below, (1) click "Outgoing WS-Security Configuration",click (2)  "add" and give a name to your configuration then and add sub-component encryption (3). (namespace is not seen on image, it is: http://schemas.xmlsoap.org/soap/envelope/)



Repeat the same for signature. Order of encryption and signature is important. Change with order keys if it is not seen as below on your screen.

Now assign the configuration to "http://localhost:8080/hi/service/certifiedUserEcho" endpoint.

To achieve that double click on XmlEchoSoap11Binding, select "Service Endpoints" and select the configuration from outgoing messages tab as seen below:


And lastly; on execution screen; select again the configuration:


I hope you followed without any problems.  Good luck!

Wednesday, April 18, 2012

Mocking Statics and java.lang.IllegalStateException: no last call on a mock available

With the help of PowerMock it is now much more easier to mock static methods or partial mock of an object (not interface).

On my first trial I recieved an error saying "java.lang.IllegalStateException: no last call on a mock available".

In fact; as stated on the document it should be enough to add @RunWith(PowerMockRunner.class) and @PrepareForTest(MyClassThatHasStaticMethof.class) annotations prior to class defination.

It is also stated on here.

But I continue to see the same error. At the end I noticed that although I was using EasyMock; I was using Mockito imports:

You should use "import static org.powermock.api.easymock.PowerMock.*" instead of "import static org.powermock.api.mockito.PowerMockito.*"

It was my fault to choose the first import recommendation by ide. Don't forget to be sure you are importing the right package.

Monday, February 6, 2012

Using .netrc with Windows for git without password prompts

These days I am working on Windows and I spent some time to learn the shortcuts I have used to use with Linux.

If you are using a git repository; you often want to declare your password somewhere in order not to enter in every pull/push.

In windows it is a bit tricky. Tricky part is that you name fie _netrc instead of .netrc. The configuration I did for my bitbucket.org account is displayed below. Another thing to keep in mind is you net to remove username from url as bitbucket suggests. In that case _netrc is not used.

Wednesday, January 4, 2012

Encyrpt in Java & Decode in Python

A small string needed to pe encrypted in java and decoded in python. Here is how we did it:

Python side:
And here is the java part: