Integration has become very easy. API's and standards like REST (ok, not exactly a standard) and SOAP mean that anyone with an IDE can integrate. It's even become cool to use API's, and that scares me.

Why fear? Well, I've been doing "integration" for a long time. It's hard. Or, rather, it's hard to do well.

It's not about the connection, which is now easy. It's about the best practices that we've developed over years to prevent problems down the road.

I wanted to share one of the immediate benefits of LinkSpan for people who might not have the enterprise experience that I have, or for people new to coding who might not yet realize how things can go wrong.

Ever since pagers came about, IT Administrators and Application Developers have been scripting alerts to let them know how their computers, infrastructure, and apps are doing. From pagers, to text pagers, to SMS, to mobile email… sys admins and developers grep log files and pass critical status messages to them so they can stay ahead of problems.

Stay ahead of problems.

Developers usually know when things are about to go off the rails. They simply script some of that knowledge, and use it to alert them so they can fix problems while they can still be resolved without too much bad happening.

Their knowledge, in "service or API terms" can be thought of as "the contract". The contract the developer has with the API they're using. In many cases, the contract is implied, so it's not something you might see articulated anywhere but API documentation. Specifically, how to use an API and what sorts of messages/formats/responses to expect.

The important thing about the contract is that it ends up being specific to the way you're using the interface (in some regard). The API may be up, but your expected results may not be happening in the way you expect (for a variety of reasons). Developers know that up/down isn't what matters (most app/system management tools and teams have that covered very well), and that it's usually a degradation of service that impacts things worst. Even worse, degradation of service is the problem that's most difficult to resolve.

LinkSpan provides an "everyman's scripting engine" for ensuring the contract you expect is the contract you're getting.

Simply connect to the API you're using (or import your custom API), pick the methods you're interface is supporting, then paste in the data format you're app expects. This is all much easier than it sounds, and LinkSpan gives you a simple wizard to walk your through the steps. That test can then be used to raise an alert, or to generate insightful dashboards and reports you can use to track trends and plan future projects.

Now, developers can go about their business, knowing that Facebook, Twitter, or whatever API your app depends on is working according the contract you expect. And, when things start to go sideways, you're the first to know and can take the mitigating actions necessary to prevent the API's "failure" from impacting your users.

Peace of mind. Easy.

Why not try LinkSpan free for 15 days?

Ask any developer what the most boring and frustrating part of programming is and odds are you will hear “testing” come up in the first couple of responses.  While it is rewarding to get through the testing of any code as the final sign of task completion, the process itself can be extremely aggravating and bewildering.  A savvy consulting services shop will allocate just as much time for testing as development and while they often have to defend such an estimate to their clients, that is experience talking not simply a desire to inflate the bill.

The other aspect of testing that makes it very boring is that it is a process that never really ends.  Just because something works correctly today doesn’t mean it will continue working tomorrow.  Not only are their often seemingly harmless code updates that can have undesirable side effects, there are environmental changes that can also have dire consequences.  I’ve seem many scenarios where a change to a router rule or a configuration update would impact entire systems.  The only way to possibly provide assurances that your application is still working as desired is through continuous regression testing. 

Many organizations and developers will simply rely on the ongoing monitoring of real-time activity in lieu of continuous testing but this will often leave holes in a true continuous testing strategy.  However, true continuous regression testing would provide independent verification that the application or system still behaves as originally intended.

This is exactly why automation and testing go so well together to provide a much higher level of assurance.  This has been realized through the many tools available for testing browser-based applications such as Selenium and TestComplete.  At OpusGrid, we this same combination can be just as useful for API providers and consumers.  This is one of the reasons we targeted API testing in the very first release of LinkSpan.  Just as you can record a set of actions in a browser and play them back, LinkSpan allows you to record a specific API call and define a schedule for that to be played back.  Those results can then be captured in a report or used to send alerts.

So if you are custom building your own test system or find yourself frequently testing APIs manually, please take a few minutes to check out our quick walkthrough video to see how Automation and Testing can make your life a little less aggravating.

Day four of Oracle's copyright and patent lawsuit against Google's Android OS brings a lesson in API design and programming that even the judge declares 'confusing.' http://bit.ly/IBHL9N

Ever tried to explain what an API is and does to a loved one, a friend or an investor?  Imagine having to explain what an API is to a jury!  What are your favorite examples of how to explain "What is an API"?

Irregardless of the how much SaaS a company uses, they're still going to have internal systems. To improve integration across internal systems companies will create internal API's. They're already doing so. And, it's becoming cool to do so even more so I think we'll see some acceleration in this space.

Internal API's need to be managed just like external API's. In fact, a good API management solution will integrate on-premise and off-premise API management tools enabling developers to manage API's seamlessly no matter where the API sits.

Mashery has an interesting article about API management that unfortunately tells just half the story. A solution that only lets developers manage external services is incomplete. Internal API's are more likely to be SOAP based than REST based (my own opinion, not a research driven observation). These internal API's will need different management tools/features, but the experience of managing internal and external API's will need to be consistent (and consistently easy to use).

A SaaS API management tool is not going to be allowed access through a company's DMZ to manage internal API's. I'm pretty liberal on security implications of enterprise SaaS deployments, but even I can't see any way something like this is ever going to happen. To properly manage an internal API from a hosted API management tool will require storing your internal API contract on the external host. Way too risky.

You might want to start by managing external API's first to ensure that you're managing the service you're getting from an external provider. But keep the endgame in mind. I believe that over time your internal infrastructure will look a lot like these external ones (if they don't already). You will want to manage contracts between teams the same way you want to manage contracts to your external providers. Doing so will foster an internet-style collaboration model between internal teams, leading to more creativity in how internal systems (and their data) are used by employees (an important and desirable result).

Be careful that you don't find yourself in a situation where you invest in a tool that provides no possible path towards a unified API management experience between external and internal API's. You'll regret it. And, by the time you do, it'll be too late.

As I read about the Instagram acquisition yesterday, I was just as struck as others at how important it was to Facebook to own the user experience of posting and sharing pictures with their friends and family.  In fact, $1B goes beyond important in most people’s minds.  Facebook does a great job of facilitating picture sharing from a computer, but Instagram had leapfrogged them in the mobile device experience which gave them obvious concern. 

5 million pictures a day are uploaded and shared using Instagram.  The most valuable part of that process is the actual sharing which is done through Facebook, Twitter and Tumblr.  What is really interesting to me is that Instagram saw an opportunity to enhance a single process and was able to turn it into a billion dollar asset.  Notice that I said enhance, not build or invent.  Users could already upload and share the very same photos with the very same people.  The innovation was an application that simply made things easier and more enjoyable for the user.   This is why it only took 12 people 2 years to grow Instagram to 30M+ users.

The key to all of this quick growth was APIs.  Instagram was able to build an application that enhanced the user experience without starting from scratch because of the FB, Twitter, and Tumblr APIs.  Because of these APIs, they only had to build out their own photo repository which is a lot easier than building your own social network.  Just ask Google   

Building from scratch is really hard and risky and its usually unnecessary.  The key to quickly enabling a valuable customer experience is to build on top of what already exists and reaching out to existing users if at all possible.  No next gen application is an island and yesterday we saw more evidence of how crucial APIs have become in enabling the next wave of innovation.  Instagram is just the latest example and certainly not the last.

Hah! No, it really doesn't.

What it does need are similar capabilities to help companies manage the complexity of integration, even when integrating using standards based service models and API's.

In a very interesting Information Week report on ERP, writer Doug Henschen makes some really interesting points that I believe is a "must read" for companies using API's as a core part of their application/IT strategy and vendors who sell to those companies.

In a section titled "the integration imperative" Doug shares a Starbucks use case. Many who look at that use case would think of it as a mobile application, or even an example of social technologies used for business. Underlying all of that are a few key things, without which such innovation would be impossible:

1. Be willing to deploy pretty good, not perfect, apps. Gen Y and Millenials are used to things mostly working, and getting them faster. As a solution provider, having a solid infrastructure and a good understanding of your data and API will let you be nimble and have short release cycles.

2. API's are a core feature requirements. This report states that 64% of respondents cited easy integration as the most important thing they look for in enterprise apps. That's a huge difference to the secondmost cited quality which came in at 38%. Implied in this, if you are a vendor, is that you deliver an API that people can use easily for complete use cases. Said in "MBA terms" - you need a "whole product offering" around the API. Not just a REST interface.

3. Develop a way to listen and respond to the user community. The report shares an example from Guess, but I'm going to share a personal one. I was once in a Kinko's getting business cards printed, and the guy helping me was clearly having a hard time with his software. Within a few minutes, I was behind the counter, pointing and clicking with him, asking him how IT decided what the software should do, how it was deployed, what's wrong with it, etc. The people that use this stuff actually know something. Now, you don't want to just deliver "faster horses" but you do want to interpret their insights and add value based on your deeper technical understanding of the possibilities. (PS That's a reference to Henry Ford who said something like "had he listened to cusotmers, he wouldn't have created the car but built faster horses.)

It goes without saying that OpusGrid can help with the three key requirements above. Whether you are a small company looking ot use other API's, a cusotmer using services inside your internal integration infrastructure (dare I call that a "private cloud"?), or a vendor trying to make sure you're deliver a whole product around your API's.

Why not check out The Right API, or drop us a line?

With the API craze that started sometime in the last 24 months, it might be easy to forget that API's are not new. Nor are the things we do with them.

There's been a lot of discussion about understanding your API. What's your business model? Who are your users? I know there are some grey hairs (and some blond-highlights) wondering what the commotion is about.

It's all about integration.

And, when it comes to integration you need an API.

I remember building networks that needed multiple network cards. You know, one for each network protocol (remember Novell Netware or Banyan Vines?). Back then, there was plenty of time to figure out the "application integration stuff" because what was happening at lower layers of the OSI model gave developers the time to react.

Then along came Cisco and others, and they consolidated the network. Yeah, there was a time there was something called a BRouter - a router that also bridged traffic.

(Forgive me, I think the heat-induced lethargy is causing me to be a bit nostalgic... reminding me of when I was young enough that the heat didn't slow me down.)

Anwyays, networks were much harder to build. You actually had to know where your users were, keep them as local as possible to the servers and the apps they were using. VPN's crushed that complexity, adding some more of their own.

The good news was, with VPN's any user could be on any network, but access their apps as if they were local. The bad news was that with VPN's any user could be on any network, but access their apps as if they were local.

It was easy to know - if a user called from the 5th floor, you'd go check out the 5th floor network. Now how are you supposed to know where that user is on your network. (Reminds me of the time a customer lost a server - they knew it was down because it stopped responding, but they couldn't find it. It took about 2-3 hours to find it. It was in a closet, which is also why it failed. There wasn't enough ventilation.)

Fast forward a bit more.

Now there are widespread standards all the way up the stack. Take a WSDL and within minutes someone without any knowledge of the target system at all, but with a reasonable development tool could be integrating systems.

Just like with the users in my VPN example... this creates some great opportunity only we've moved up the stack, so instead of talking about users, we're talking about data. Now, the good news (and the bad news) is that data can be anywhere.

What data? Well, any data. Let's say - customer data. Well, what makes up customer data?

Address. Phone number. You know, customer data.

What does it look like? How is it stored (in each an every system you're working with)?

I know what you're thinking... and let me tell you - Yeah, it does matter. And, it can be complicated.

Let me give you an example.

I upgraded to OS X Lion (for my Mac) this week. I went into iTunes, clicked "buy", and Apple asked me to validate my identity before allowing the purchase. It brought up a panel that asked for my credit card info, name, address, and phone number.

Simple, right?

I was on support for 15 minutes, frustrated, when I figured out the solution.

I logged into American Express, looked up my credit card information, and saw that on my American Express account, my apartment was listed simply as #37F, in a separate apartment field.

Now, I happen to know that the USPS standard for writing addresses is to put the apartment number on the first line, no comma separating. Something like '40 Harrison Street APT 37F'.

When I matched what I put into iTunes to what American Express showed on my account profile, bingo-bammo, it went right through.

Are you doing the integration dance? This isn't some leading-edge SMS-map-real-time-mobile-startup. This is Apple and American Express.

What's going on?

They use different data models. Neither particularly adhering to the Post Office's standard. Remember, it's not just Apple and American Express. It's Visa. Discover. And, so on. And, it's Spotify, Google, Amazon and every other service.

So what do you do? I mean, besides make your users hate you?

Manage your API properly. Deal with the trust, visibility AND data issues in a systemic way. API providers and consumers (both internal- and external-consumers) will have the ability to manage your data model, easily map that data model to the API's that make up your virtual integration infrastructure, and manage your user experience in a way consistent with your brand and user-experience goals.

Yeah. Bet that title got your attention.

I'll be gone for the next 10 days and didn't want you to forget me.

My soapbox about complex integration being ported from the enterprise to the cloud being a huge opportunity for the right solution had me thinking.

I wonder if a lot of the people developing "apps" using APIs even realize what governance is, and why it's critical. At a super high level, governance is the ability to articulate what to expect from an API, the formalization of that expectation into an SLA (service level agreement), and the careful monitoring of the actual API to make sure it continues to meet the SLA as defined.

Of course, one API may have many SLAs based on usage, and all that.

So, where does online dating come in as a benchmark?

I had this idea for turning Facebook into an online dating site to solve a huge problem with online dating. Let's think about the problem with the traditional online dating model.

A person goes online and tells some random website a whole bunch of personal details. Without any confirmation or reference at all, their profile is then published for other people to view. People seem surprised at all the exaggerations that are exhibited in personal profiles.

Like money, dating is something people don't seem to have such realistic views of when it comes to their situations. Makes sense too... if I met you for real, I wouldn't start the conversation by saying "Hi, I'm a compulsive neurotic who will make your life miserable." I'd more likely say something like "I'm great at my job because of my attention to detail" if I even let it come up at all in conversation.

While that's sorta funny... there are other more discrete ways to define people that aren't so "fuzzy". Like, "I'm not married" or "sure, this picture was just taken last month".

Married. Current photo. These are bits of meta data about the person that could be easily confirmed, though we're all concerned about big brother and (US) law doesn't make it easy for companies to be so intrusive in our life.

We're so embarrassed by online dating, that we keep it separate and only let other daters see dating profiles. So, we can exaggerate because our "regular" (non-datable) friends won't see what we're saying or hold us accountable.

From a technical perspective, the same analogy means that sometimes the technical infrastructure isn't there to constantly prove that what I'm saying about my API is true - even if it's a discrete measure. Like uptime. Or, expected performance.

And, that's where Facebook, or rather the community comes into play.

Imagine building a dating site layered on top of Facebook.

All of a sudden, when I say I'm 6'1" and my mother says "in heels" (that's funny because I'm a boy) someone looking at me as a prospective date will know my real height. If I say I'm not married, I'd have a hard time when I got home that day from work. And, if I thought I could create another profile, I'd need to figure out a way to get a reasonable # of friends so I look like a real person, with all the interactions on my facebook page that entails.

The community could hold people accountable.

Back to APIs.

However, if you'd like better governance of API providers, a way to let the community hold companies to a higher ethical standard of how they present their services, wouldn't it be better if the community had a place to share their experiences of the real level/quality of service provided by an API vendor so you could choose the right API for your situation? This community should be in the open, available for prospective customers, but validated by existing ones. Then, we'd have something that more reflects reality and everyone would prosper as a result.

It's absurd... and I'm shocked by such raw behavior.

Of course, maybe I'm old-fashioned? I've been feeling my age a bit lately, so it could be.

Maybe, I'm just too hard. I've seen a lot. Did you know that I once created a song for an IT department, so that they'd remember how IP multicast worked? Yeah, that was pre-internet. Well, truth-be-told, pre-internet-as-we-know-it.

And, that's the rub.

I know things. I've seen things... NO! Don't ask. I won't tell.

What I will say is, be careful.

When you look under the covers, stick your nose into places... you'll never look at a computer application the same way. Never speak to a customer service person without wondering. Never. Book. An. Airline. Ticket. Online. Without. Wondering.

Have they used a vendor's native API for integration without creating a layer of abstraction to prevent vendor lock-in?!?!

There, I said it.

Best practice numero uno. Number one. ABOVE ALL THINGS.

Don't code to a vendor's API because you're stuck with whatever they decide to do with it. You've got their methods, and methodologies (what I like to call "integration philosophy") baked right into your application. And, if you diverge from that vendor, philosophically or commercially, you're screwed.

This is extra important when you have people in your organization using external API's. Do you even know who they are? That's problem #1. Make sure to build in a gateway layer, or intermediary layer, whatever you want to call it. The important thing is to decouple your consumers from those API's so that you can operate independently from the vendor, their methodologies and their roadmap, and switch more easily between vendors should you need to.

Good luck, and code safely.

My thoughts this morning are on chaos. Complexity.

Integration.

What's it all about?

Integration.

The single biggest challenge with integration?

Choice. And, the rapid pace of change.

I know, pick one.

I can't.

There are too many options.

Reading an article on Bloomberg News this morning about Groupon's IPO this morning, I saw that Groupon has 482 imitators. Groupon launched in November 2008, roughly 30 months ago. That means, since launch, roughly 16 competitors a month have entered the mix. Keep in mind there are about 20-22 work days a month. That's close enough to 1 a day as to be really scary.

How do you make a good choice with so many options. Modern thinking on happiness says we can't. Too many choices are bad for happiness.

I think they're bad for business too.

And, when you combine all that choice with the pace of change you have situation where people can't keep up. It's like drinking from a firehose every day just to stay even. And, for many people this "integration" thing isn't even their full time job.

What do you think happens when people are presented with tons of choice, and a constantly changing landscape?

They don't choose.

That's right. They choose not to make a choice. And that's bad for your business.

Transparency is a market imperative, and of course, we each have our own interests. Some of us want transparent performance, others security. Some want transparency into how underlying algorithms are calculated, some only care about the response. In any case, transparency is "free market lubrication".

When it comes to integration, a highly complex collection of technologies to solve some business purpose, transparency into what's actually happening, and what can be expected to happen in the future, is critical to long-term success of the API market.