Monthly Archives: July 2010

02
Jul

Integrating MS Project with an Enterprise Product

Software Product Development July 2nd, 2010

This blog deals with integration of MS Project with an Enterprise product

At a very basic level, MS Project, or MSP, is a project management tool, used by several organizations across various sectors to track and manage projects.

As a real-life example of the value that MSP can add to a product’s development phase when properly integrated with the development cycle, an interesting article is given next.

In a particular project, the client had developed an enterprise product to track projects, which were part of Six Sigma initiatives within the organization. Over the years, this product was transformed into a tool, which could be used to manage projects within multiple initiatives such as IT, new product development etc.

This change in the product’s roadmap created a need to integrate the product with MSP. A primary reason for this was that a lot of customers were of the view that their managers were quite comfortable using MSP and wanted to continue using it as a scheduling tool.

Plug-in has to be installed on each machine where they want to use MSP and integrate with product managed under MSP. This plug-in will then synchronize with the MSP APIs to set and retrieve data.

To develop the plug-in, the following steps were taken:

1)    First, the language in which plug-in was to be developed was decided upon. A Microsoft language was preferred, since it would provide in-built objects to access MSP APIs. In this case, Visual Basic was selected since it has fewer compatibility issues.

2)    The plug-in would need to primarily perform two operations:

  1. Import Data into MSP: By reading the source data in a specific format (in this case, XML), the plug-in would need to set the data using the APIs provided by MSP. This includes creating resources and tasks, and setting dates, efforts, constraints, resource rates etc. This is a critical step since the sequence in which the values of the fields need to be set has to be understood. For example, every task has actual work and actual overtime work.  When these values need to be entered into MSP from a host system, both of them first need to be set to zero followed by setting the actual work and the overtime work values, in that sequence. The sequence was critical here, because otherwise, incorrect values would be entered in MSP.
  2. Export Data from MSP: This step would be needed to get the data from MSP and generate a file, in the format expected by the enterprise product. As compared to the data import process, this was a relatively easier step, although certain cases could require a connection to some web service exposed by the enterprise product followed by sending the data in a file, which is generally in the XML format. An alternative would be to create a file and then ask the user to import it into the system through the enterprise product.

3)    The next step was to create an executable installable for the plug-in.

4)    The similarities and differences between MSP and the product also had to be examined as it was critical to build the logic based on how the product and MSP worked. There could be no single solution, which would integrate any product with MSP as the latter is highly customizable and has to be configured keeping the product in mind.

5)    The version of MSP being used was also important since MSP 2007 has some advanced features, such as “Resource-Cost” as a type of Resource, which are not available in MSP 2003.

So as illustrated by the example above, the reader may now agree that integrating MSP with any enterprise product would certainly add a lot of value to the product. To be able to do this successfully though, one has to have a good knowledge of MSP and be aware of its various configurable parameters, only after which, a plug-in can be developed.


01
Jul

Public API – Points to consider

Product Ideation July 1st, 2010

In this blog, we will look at various factors to be considered while  designing Public APIs

The web has moved from its image of hosting web applications to that of a platform. A platform is primarily, a set of building blocks that you can combine in innovative ways to build your own application. One of the key factors that have driven the web towards the “platform” avatar has been the emergence of a large number of Public APIs based on Open Standards. A Public API in simple terms is exposing key pieces of your application for the consumption of other applications.

A Public API is an important piece of any web application today. Over the last 5 years, public APIs have proliferated and everyone from the big companies to the newest startup bets big on a public API. According to the premier API tracking site, The Programmable Web (http://www.programmableweb.com), there are currently 2033 APIs available for applications today [dated 21st June 2010]

What can we do with the Public APIs? Here are some of them:

  1. You can integrate a best of class functionality that is already being exposed by the Public API. For e.g. if you need to integrate Maps, simply integrate the Google Maps API. You do not need to reinvent the wheel.
  2. You can provide a complete different user experience (interface) to an existing service.
  3. You can utilize the Public API in a larger mashup that you are creating.
  4. You can create applications running on devices that the current provider of the Public API does not address. For e.g. you could write mobile applications running on the iPhone, Android, etc – which the Public API provider does not even provide.

Now that we have discussed some of the points about what one can do with a Public API, here are some points about potential benefits that a public API brings to your existing application.

A Public API gives access to your platform to a huge number of programmers/applications.

  1. Using open standards, they are able to build out applications using your API in ways that even the creators of the API could not have envisioned. And since the public APIs are usually built on Open Standards, they are integrated into a wide range of applications that run on hardware ranging from desktops to mobiles.
  2. In the current market scenario, programmers do not want to build something that is already exposed by the publicly available API and if the API is backed by a scalable and high available environment, they will adopt it.

This article now builds on the premise that you are convinced that a public API will help your already available service. The article now discusses important points that you must consider before you embark on coding out the public API. Each of these points focuses on a theme rather than the implementation details and they are listed in no particular order of importance.

  • REST v/s Web Services (SOAP)

These are the two predominant models of invoking your API: REST and Web Service. REST is being preferred by most applications today, primarily because of its simplicity and implicit mapping to operations like GET, PUT, POST, etc. It is fair enough to say at this point that you should expose your services via the REST way, even if you have a Web Services (SOAP) mechanism of exposing your services.

  • Response Data Format (XML, JSON)

Every API call results in data being returned to the calling application. The data could be a result contains data records or in case of update APIs, simply a response text indicating “success” or “failure”. Two formats are predominant here: XML and JSON format. Your API should be flexible that it will allow the caller to specify which format they would like the data response to be sent back. Several APIs of late have been adopting only JSON as the supported data format. The best option over here at this point in time should be to support both if possible.

  • Service Contract Description

A Service Contract of your API is extremely important. It clearly defines what features your public API provides. This includes:

  1. The different Services and their names
  2. How they are accessible
  3. Authentication mechanisms
  4. Method signature for each method in a Service
  5. Description of each parameter in a method

It is important that you take small steps in the initial release of your API. Do not expose all functionality at once. A preferred approach might be to expose a read-only API that first gives users access to your data and then introduce write-methods that allow users to save their information. Introduce newer methods/functionalities in later versions but release a fairly compact Service Contract in your initial versions. It will help you identify the chinks early enough and help to refactor the existing Service Contact and introduce newer ones.

  • API Authentication

One need not stress enough about how important it is to make your public API secure. You only want authenticated users to access your public API. There are various strategies and authentication/authorization standards emerging for you to begin with. The most common approach is to issue a API Key for any application that wants to use your public API. This API Key is provided on signing up and is to be provided with every request to your API by the calling application. You can then authenticate the key and also determine if the call is within acceptable Rate Limits (this is covered in a point later) that you have set per API Key.

An authentication standard OAuth is gaining widespread acceptance as a preferred way to allow users to first validate themselves with the Service Providers and then pass a token to the calling application that they have been validated. This removes the need for the calling application to store your users’ username/password with them. A large number of popular public APIs like Twitter have adopted this standard and you should have it on your radar.

  • Service Versioning

As your service evolves, the public API could also change to match the new features. You need to make sure that you do not break existing clients who have already binded to your existing public API. Enter versioning! Think about versioning your public API right from the first release. If you are using a REST like mechanism that inserting a version number like 1 or v1 might be a good start. With each version, specify clearly what is different, what the additional features are, etc.

  • Rate Limits

All public APIs should be free to exercise to increase their adoption in the initial phase. Unless you strongly feel that you need to have a paid public API right from Day 1, you should definitely make the public API free to use. However, it is important for it to make business sense at the end of the day. After all, computing resources do cost. An approach typically used by almost all companies is to build in Rate Limits (or Quota Limits) into their public API. Some examples include:

  1. 10,000 calls per day
  2. 1000 calls per hour
  3. 1 GB Free total disk space (This is required in scenarios where you also persist the objects in your cloud as part of the public API)

The rate limits are typically reset at regular intervals like an hour, daily, etc. In certain cases, where you also provide storage space, the disk space quota is fixed at an upper limit and is not typically reset.

No matter what your functionality is, defining the rate limits is important because you will need to build checks in your public API infrastructure.

At the same time, there is a clear possibility that if your application and consequently your public API, is huge successful, then there will be applications which will easily go past your rate limits. That is a good problem to solve. In those scenarios (and you should expect them), you need to think about any of two approaches:

  1. Selectively increase the rate limits for a particular application. You will first need to discuss the requirements with the original creators of the application.
  2. Provide a paid option, where the application buys the increased limit either in tiers that you have set up or on a pay-as-you-go service. The pay-as-you-go service is generally preferred because you may not get the high spikes every single day.
  • Documentation
    It is important to support your public API with strong documentation. Pay attention to every aspect of the documentation. At the minimum, you must document the following clearly:

    1. End Points for different Architectural Styles (REST,WS) along with versioning.
    2. Define all services, methods (actions) and their parameters
    3. Provide sample request / response data formats for each method(action) in your service
    4. Document the error scenarios clearly along with Server side status codes

The best way to get started in documentation is to look at the documentation of existing public APIs. Pick one that suits you and stick to it. Several public APIs even have a RSS Feed about their API documentation to inform about any changes.

  • Helper Libraries

It is important to envelop your REST/Web Service calls into easy to use helper libraries. Helper Libraries target a number of client programming languages/platforms like Java, Python, Ruby, Javascript, .NET, etc. A Helper library gives a quick starting point to get someone exercising your public API within minutes or hours. A Helper library should address the following (recommended):

  1. Envelope the security calls (Authorization)
  2. Envelope the REST/WS calls and Data Format (XML, JSON) parsing.
  3. Optionally, return the results in Classes defined in the target high level language. For e.g. Java classes, etc.
  • Dedicated Public API Web page/site

It is strongly recommended to have a companion web site for your Public API. This companion website can serve as a one stop destination for all documentation related to your public API. It can also provide information like:

  1. Signing up for the API and getting issued an API Key
  2. Showcase applications / case studies of how people have used  your public API
  3. List official Helper libraries and contributed Helper libraries that make it easier to use your API.
  4. An API forum where users discuss your API and which your API support engineers can regularly monitor
  5. RSS Feeds that allow people to subscribe to API updates. Providing a “Sign up” for email updates to notify users is also recommended.

I hope this article gives a broad framework about the high level factors that you can consider your Public API release to support. Designing and Development can then commence much more smoothly. There are many other small details to consider, so please share your additional points in the comments for the benefit of all.

References

  1. ProgrammableWeb (http://www.programmableweb.com)
  2. Open APIs: State of the Market, May 2010. John Musser (http://www.slideshare.net/jmusser/pw-glue-conmay2010)
  3. OAuth Protocol Site : http://oauth.net/