When ZapThink last wrote about Business Process Management (BPM) in the Cloud in March 2012, we challenged both vendors and BPM customers to rethink their approach to BPM software, eschewing a heavyweight middleware approach for the lightweight, hypermedia-oriented approach that Representational State Transfer (REST) encourages. And while we did generate some short-lived buzz, most of the response – or lack thereof – was little more than a resounding silence.

True, work on Cloud-friendly, REST-based BPM continues in certain dusty corners of academia, most notably in the research of Cesare Pautasso, a professor at the University of Lugano in Switzerland. But in spite of his notable contributions to Thomas Erl’s SOA with REST book, the enterprise software and Cloud marketplaces have largely either ignored or misunderstood his research as well as ZapThink’s on this topic.

While it’s amusing to theorize a vast vendor conspiracy, positing middleware dinosaurs actively working to distract their customer base from lighter weight, Cloud-friendly approaches, the reality is likely to be far more mundane. People just don’t get it. Or to be precise, our audience doesn’t get how all the pieces—BPM, REST, Cloud, and even a bit of SOA—fit together. To help resolve this confusion, let’s resort to an age-old technique: let’s draw some pictures.

Framing the Cloud-Friendly BPM Problem

Let’s start this discussion with an illustration of an admittedly simplistic business process involving one person and some back-end system, as shown in Figure 1 below.

Figure 1: Simple Two-Tier Process

Note that in the figure above, the user might tackle a few tasks, and then the server takes over, executing a few tasks on its own. While the server is busy doing its thing, the user might query the server as to the current status of the process.

So far so good, but we don’t want our server to serve only one user at a time. After all, the whole point of the client/server pattern is that it is many to one. As a result, we need to introduce the notion of a process instance. For the sake of simplicity let’s assume that we don’t have more than one person participating in a particular instance at the same time. But we might have multiple people each running their own instance of a process, for example, completing a purchase on a Web site, as shown in Figure 2 below.

Figure 2: Two Tier Process with Instance

In the figure above, the BPM engine running on the server spawns a process instance to deal with the interactions with the user. If multiple users initiate the same process, the server can instantiate as many process instances as necessary, and the engine keeps track of where every user is in their instance—in other words, the instance state.

How to keep track of all this state information in a scalable, robust manner is at the core of numerous distributed computing challenges. Today’s BPM engines generally run on Enterprise Service Buses (ESBs), which maintain state by spawning threads—short-lived, specialized object instances that run  in the execution environment of the ESB. But while threads are short-lived, process instances might take days or weeks to complete, and furthermore, threads are specific to the execution environment, making cross-ESB processes difficult to implement. For these reasons, we call state management the Achilles Heel of traditional, heavyweight (Web Services-based) SOA.

If such ESB-centric issues weren’t bad enough, the Cloud introduces a new wrinkle. Because we want to run our server in the Cloud, we don’t want to use it to maintain any state information, because we expect virtual machine (VM) instances to fail. In the Cloud, we provide automated recovery from failure rather than avoiding failure. However, if we store all the state information in the underlying persistence tier (not shown), then we limit our scalability, since every time anyone clicks a link, we must update a database somewhere.

What we need is a better way of dealing with state information that both allows our BPM engines to be Cloud friendly, and also frees us from the limitations of our ESBs. Or perhaps we must reinvent our ESBs to work in the Cloud. However you slice the problem, Hypermedia-Oriented Architecture (HOA) has the answer.

HOA to the Rescue

As ZapThink has discussed before, many people misconstrue REST as an API style that features a uniform interface, where in reality it’s a style of software architecture for building hypermedia systems. Why is the latter definition a better one? Because Roy Fielding, its creator, says so. That being said, work continues on the architectural context of REST, perhaps extending Fielding’s original thinking, as well as beyond the API style that most techies think of when they think about REST. We call this extension of the REST architectural style Hypermedia-Oriented Architecture, or HOA.

The central principle of HOA is the HATEOAS REST constraint: hypermedia is the engine of application state. In essence, HOA separates two different types of state information: application state and resource state. Application state corresponds to the user’s place in the runtime workflow consisting of hyperlinked representations, while resource state remains on the server, keeping track of persisted state information and state information that multiple users share.

On the one hand, HATEOAS requires hypermedia to manage all state information specific to individual clients, and on the other hand, delegates all other state information to the server. REST also specifies a set of verbs for querying an changing state information: GET for querying resource state without changing it,  and three verbs that change the resource state: POST for initializing a resource, PUT for updating a resource, and DELETE for deleting a resource (assuming we’re using HTTP as our transport protocol).

Note, therefore, that all verbs other than GET change the resource state, while all verbs, including GET, change the application state. Furthermore, all state information appears in the messages between client and server: the requests from client to resource, and the representations from resource to client. By extension, HATEOAS requires us to only use POST, PUT, or DELETE when—and only when—we must update resource state.

With this principle in mind, we have a real problem with the process in Figure 2. Note that the server is maintaining application state, which HOA forbids. But we can’t solve this problem simply by picking up the process instance from the server and sticking it in the client and expecting it to work properly, because sometimes we really do want to update the resource state. We somehow need to separate the process instance into two (or more) pieces so that hypermedia on the client can be the engine of application state while the BPM engine remains the engine of resource state.

Figure 3 below illustrates this principle. The client sends a POST to the server, which initializes a resource. In this case, that new resource sends a hypermedia representation to a stateless intermediary which caches the representation. This hypermedia representation is essentially an abstraction of a dynamic set of hyperlinked representations, for example, one or more php scripts that can generate a set of hyperlinked Web pages. Once the intermediary has the hypermedia representation, it returns the initial representation (for instance, a Web page) to the client.

Figure 3: HOA-Based Process with Stateless Intermediary

From that point on, as long as the client is navigating the application via hypermedia, changing only the application state as the user moves from one step in the process to the next, there is no need to change the resource state—and thus, no further POSTs, PUTs, or DELETEs are allowed. The client may perform a GET, because GETs change only the application state. The intermediary may be able to handle the GET on its own (if the necessary information is resident in the cache) or can turn around and perform a GET on an underlying resource, if necessary.

Furthermore, the application state may change without any interactions with the intermediary or the server by leveraging programmatic capabilities on the client. If the client is an arbitrary piece of software then this capability is trivial. But even if the client is a browser, it’s possible to change the state of an application without fetching anything from the server. In fact, there are many was to accomplish this feat.

Sometimes, of course, a hypermedia application, which we might also call a HOA process, must update resource state, for example, when it’s time to process the user’s credit card or change the number of widgets in inventory. Then—and only then—do we perform a PUT.

The most important characteristic of the process in Figure 3 is the fact that the intermediary is entirely stateless. If for some reason the VM that is hosting the hypermedia representation that is serving the client crashes, the Cloud environment must simply spawn a replacement and reload the same hypermedia representation as before. The client won’t lose its place because the hypermedia on the client are maintaining the application state. Similarly, we can horizontally scale the middle tier however and whenever we like. Instead of one VM hosting a particular hypermedia representation, we could have two or a hundred, and it doesn’t matter which one responds to a particular GET from the client.

Combining HOA Processes and Traditional BPM

The problem with the example in Figure 3, of course, is that every client’s process is separate from every other client’s process. However, most business processes in today’s organizations involve multiple parties—either multiple people or multiple enterprise applications or some combination.

On first glance, HOA doesn’t address such complex processes, since HATEOAS only deals with application state, not resource state. Fortunately, HOA works perfectly fine in this broader context as well, because it calls for a separation of application and resource state while providing for multiple ways to update resource state. After all, POST, PUT, and DELETE all update resource state, and any user can execute these verbs for a particular resource. Figure 4 below illustrates this more complex process.

Figure 4: HOA Process with Composite RESTful Service

In the figure above, a POST from a client instructs the BPM engine to instantiate a process instance on the server as in Figure 2. The first step in this process creates a hypermedia representation for the client to interact with as in Figure 3. Meanwhile, the resource state may change via any event, including a server-generated event or the action of a different user. If a user executes a PUT on the client to the hypermedia representation on the intermediary, then that representation turns around and PUTs to the appropriate underlying resource. Or perhaps the client PUTs to an underlying resource directly. Either way, the PUT goes to a hyperlink the client obtained from a previous representation at an earlier step in the process.

We might call the process running on the server a Composite RESTful Service, because the intermediary may abstract the entire server-based process via one or more RESTful URIs. A simple example of a Composite RESTful Service is a chat window application. Multiple users share the same chat session, so clearly the chat session state is part of the resource state.

There are a few essential points to keep in mind about the illustration in Figure 4. First, the intermediary remains stateless and therefore Cloud-friendly. We must maintain resource state in the persistence tier, but since we’ve offloaded the maintenance of application state to the client, we won’t be overburdening our database. We may also interact with our Composite RESTful Service via RESTful interactions, an essential benefit that Prof. Pautasso emphasizes in his research. And finally, not only is the middle tier horizontally scalable and elastic, so is the client tier—because every user brings their own client to the process.

The ZapThink Take

With the addition of an appropriate approach to building a RESTful Service abstraction, Figure 4 also serves as an illustration of how to implement RESTful SOA, what ZapThink refers to as “next generation” SOA in our Licensed ZapThink Architect (LZA) course as well as in my new book, The Agile Architecture Revolution. We therefore have a single, simple diagram bring together the worlds of SOA, BPM, Cloud, REST, and HOA.

The secret to getting all these architectural trends to work well together centers on how we deal with state information. We must first separate application state from resource state, and then subsequently take the conceptual leap to understanding that the best way to implement our business processes is by combining HOA processes with Composite RESTful Services. Once we make this leap, however, the pieces of this complicated puzzle finally fall into place.

Image credit: Bruce Guenter

Before I joined ZapThink in 2001, I was an analyst at IDC. I wasn’t there for long, but I did manage to attend their analyst training course. Perhaps the most intriguing lesson I learned from this boot camp was how IDC goes about building their market models, which are the central tool they use to predict the growth of markets. Essentially, the market model is an immense Excel spreadsheet, with a separate row for the revenues for each vendor in the entire IT universe.

Group vendors into separate, distinct markets (or do so for individual products, when vendors compete in multiple markets). Assign each of your analysts to interview each vendor once a year in the analyst’s assigned market, in the hopes that the vendor will reveal their true revenues for the year in question. Put the resulting numbers in the spreadsheet, add them all up, and you are able to calculate a reasonable estimate for the size of the given market.

Predicting the future growth of a market, however, is another story. Here’s the secret: select all the cells for a given market for the last couple of years and drag to the right. Voila! You’ve just predicted the market for however many years into the future you care to estimate. That’s all there is to it.

In essence, this drag-to-the-right technique extends the current annual growth rate (CAGR) into the future for each company in the model. Of course, some will grow faster than others, some will prevail and others will fail miserably, but the theory is that such variations will balance each other out. Next, publish your predictions for, say, three years out and watch the press, VCs, and startup entrepreneurs whip themselves into a frenzy.

The Pitfalls of Predicting the Cloud Market

The primary weakness of the drag-to-the-right crystal ball, of course, is the old adage that past performance doesn’t guarantee future results. Throw a global recession into the mix, say, and you might as well chuck your spreadsheet and start over. But there’s an even greater problem with predicting a market like Cloud Computing. Because Cloud is an emerging market, the big spreadsheet market model doesn’t work at all. Here, then, are the pitfalls to avoid:

• Emerging markets diverge. Take the Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS) markets. Instead of remaining as three distinct market categories, vendors love breaking out new variations: Integration-as-a-Service, Big Data-as-a-Service, Identity-as-a-Service, and so on. Each of these splinter markets might be considered a subset of one of the three core service models, until such time as the marketplace itself decides otherwise. The same divergence has struck the Public vs. Private Cloud markets, as players break out separate Hosted Private Cloud, Enterprise Public Cloud, and Virtual Private Cloud markets.
• Emerging markets overlap. No one would disagree that Amazon Web Services (AWS) is an IaaS player. But what about, say, AWS Elastic Beanstalk? That’s PaaS. Or Amazon Payments? Yup, SaaS. So do you include the revenues for those individual services in the IaaS number or not? Such overlaps are slippery and dynamic, and throw a wrench into any kind of spreadsheet-type number crunching.
• Emerging markets evolve. Remember the Business-to-Business Integration (B2Bi) market from the 1990s? Well, it’s still around, but now we call it Integration-as-a-Service. Or perhaps Cloud Integration. Oops, wait a minute: now B2Bi is actually a function of a Cloud Service Brokerage (CSB). Only the market hasn’t really figured out what it wants a CSB to be yet.
• Vendors play games. Vendors will call their products anything they think you’ll shell out money for, and they’ll jump on any market bandwagon they think has momentum. Back in the 1990s nobody wanted to pay for software to manage application programming interfaces (APIs), but then in the 2000s the Web Services market took off, and a few dozen vendors started calling themselves Web Services Management vendors. It didn’t matter that Web Services are nothing but standards-based, contracted APIs. But then SOA caught the attention of the market so the same vendors became SOA management vendors. Then as companies struggled with SOA, the selfsame vendors reinvented themselves as players in the SOA governance market. Now that SOA is old news, what do they call themselves? API management vendors!
• Analyst firms collude with vendors. It’s a poorly kept secret in the IT marketplace that Gartner’s magic quadrant is pay-for-play. This collusion is especially blatant when an emerging market is involved. In fact, vendors occasionally pay Gartner to invent new markets so that the vendor can be the magic quadrant leader. This approach worked for the Enterprise Service Bus (ESB) market, but didn’t fare so well for the Integrated Services Environment (ISE) market. You’ve never heard of the ISE market? Precisely my point.
• Sometimes emerging markets overlap an established market. Microsoft recently reported a billion dollars of Cloud revenues, but this number reportedly included revenues for various software licenses for customers who said they were going to use them for Private Clouds. Does that mean that Microsoft isn’t counting those revenues in their software revenue numbers? Either way, Microsoft is being disingenuous: either they’re double-counting such revenues, or they’ve shifted numbers from a sleepy, traditional market to a hot emerging market simply to inflate their Cloud numbers.
• Sometimes emerging markets aren’t markets at all. ZapThink clarified this issue for the SOA market in our seminal 2004 Service Orientation Market Trends report. At that point in time there was a clear distinction between ESBs and traditional middleware: ESBs supported Web Services while traditional middleware did not. But as we correctly predicted, it would only be a matter of time before all middleware supported Web Services. Sure enough, today there really isn’t a “non-ESB middleware” market category. In other words, the SOA market has become the enterprise middleware market, or perhaps there is no such thing as a SOA market at all, depending upon your perspective.

Cloud Computing is on the same trajectory. It’s not really a market or set of markets at all. It’s part of a paradigm-shifting trend that is reinventing how organizations large and small purchase, provision, utilize, pay for, and think about IT resources. Eventually, everything will be the Cloud, which from the market sizing  perspective means that nothing will be the Cloud.

The ZapThink Take

The industry analysts’ market sizing crystal ball may be shinier than yours, but look inside, and it’s no clearer than anybody else’s. Predicting the future, after all, is a fool’s game, especially when something as dynamic and multifaceted as an emerging market is involved. Face it: market size predictions are almost always wrong, the only exception being when the analyst simply gets lucky.

Nevertheless, for several years at the beginning of the 2000s, ZapThink gazed into our own crystal ball and published market size predictions. We were the first in line to tell anybody who’d listen that we were really just guessing, just like anybody else in the clairvoyance business. But it didn’t matter. Our customers, the press, and the broader ZapThink audience still wanted our numbers.

Fundamentally, people don’t want analyst market size predictions because they believe the numbers are accurate. People want such guesses because they need a prediction from a disinterested third party they can use in their business plans, or their annual reports, or their VC pitches. In fact, very rarely does anybody ever go back to check old predictions to see how well they turned out, because  it doesn’t really matter. Predictions aren’t really about the future at all. They’re about the present.

The moral of this story, of course, is to take any predictions about the growth of the Cloud Computing market with an immense grain of salt. It’s impossible even to define what the Cloud market really is, let alone make a reasonable prediction as to how it’s going to grow. If you need a juicy number for your VC pitch, then fine—but remember, VCs understand the vagaries of this fortune teller game all too well. Suffice it to say, however, the Cloud is here to stay, and will only grow in significance, even though putting a number on its growth is at best a wild guess.

Image credit: Chris Gladis

In my role as a globetrotting Cloud consultant, I continue to be amazed at how many executives, both in IT and in the lines of business, still favor Private Clouds over Public. These managers are perfectly happy to pour money into newfangled data centers (sorry, “Private Clouds”), even though Amazon Web Services (AWS) and its brethren are reinventing the entire world of IT. Their reason? Sometimes they believe Private Clouds will save them money over the Public Cloud option. No such luck: Private Clouds are dreadfully expensive to build, staff, and manage, while Public Cloud services continue to fall in price. Others point to security as the problem. No again. OK, maybe Private Clouds will give us sufficient elasticity? Probably not. Go through all the arguments, however, and they’re still dead set on building that Private Cloud. What gives? The true reason for this stubbornness, of course, is the battle over control.

Thinking Like a Control Freak

IT executives in particular have always been control freaks. Our IT environments have been filled with fragile, flaky gear for so long that we figure the only way to run the IT shop is to control everything, grudgingly doling out bits of functionality and information to business users, but only when they ask nicely.

But this old mainframe reality has been fading for years now. The move to client/server to n-tier to the Internet and now to the Cloud are all exercises in increasingly distributed computing, with special emphasis on the distributed. As in distributed control.

The technology powers that  be in the enterprise have been fighting this trend kicking and screaming, of course. But they’ve been fighting a losing battle. We saw the tide turn in the first-generation SOA days of the 2000s, when the IT establishment invested tried to implement SOA by buying ESBs, centralized pieces of middleware that purported to run the organization. But too many enterprises ended up with multiple ESBs and other pieces of middleware, since of course every manager in every department silo needs their own, because they all crave control. So the doomed SOA effort became a futile exercise in middleware-for-your-middleware, as the desired agility benefit sank beneath waves of rats-nest complexity.

What’s really going on here? Why do executives crave control so badly? Two reasons: risk mitigation and differentiation. If that piece of technology is outside your control, then perhaps bad things will happen: security breaches, regulatory compliance violations, or performance issues, to name the scariest. The problem is, maintaining control doesn’t necessary reduce such risks. But if you’re responsible for managing the risks, then the natural reaction is to crave control.

Managers also believe that whatever it is they’re doing in their silo is special and different in some way. So there’s no way they can leverage that shared piece of middleware or shared SOA-based Services or multitenant Cloud. If they did, they wouldn’t be special any more. Having a differentiated offering is essential to any viable market strategy, after all. So clearly my technology has to be different from your technology!

Chaos vs. Control

The Cloud, as you might expect, shakes up both these considerations, because the Cloud separates responsibility from control in ways that we’ve never seen before. Every manager knows that these two priorities often go hand in hand, and under normal circumstances, we prefer them to go together, because  the last thing we want is responsibility without control: the recipe for becoming the scapegoat, after all. With the Cloud, however, we can maintain control while delegating responsibility to the Cloud Service Provider (CSP). The CSP is responsible for ensuring the operational environment is working properly, including the automated management and user-driven provisioning and configuration that differentiate Cloud Computing from virtualized hosting. However, the CSP has delegated control over each customer environment to that customer.

By turning around this control vs. responsibility equation, we’ve placed the CSP into the scapegoat position. As long as we have an iron-clad Service-Level Agreement with our CSP, we can trust them to take responsibility for our operational environments, and if anything goes wrong, we can hold them responsible. But the control over those environments remains with us, the customer. Once enterprise executives realize this new world order, they will run as fast as they can away from building Private Clouds. After all, if you can maintain control while delegating responsibility, why would you ever want responsibility? Responsibility gets people fired, after all.

Shifting responsibility to the CSP also helps to resolve the regulatory compliance roadblock that so many executives point to as the reason to select Private over Public Cloud. A combination of a properly responsible CSP combined with a sufficiently detailed SLA can go a long way toward indemnifying organizations against compliance breach risks. Remember, regulations rarely if ever specify how you must comply, only that you must. It’s up to you (and your lawyers) to decide on the how. As long as you’re diligent, conscientious, and follow established best practice, you’ve mitigated the bulk of your noncompliance risk. The CSPs are chomping at the bit to take this responsibility, so the smart risk mitigation strategy is shifting toward the Public Cloud.

The Price of Differentiation

The second threat to centralized control of IT is the business driver toward differentiation. Whatever our department or business is doing is special and different, and thus our infrastructure as well as our application environment must be unique as well. This principle is always true up to a point, which is why executives love to cling to it like a floating log in a vast sea of change. But just where that point falls continues to shift, and has shifted further than many people realize.

No enterprise would dream of calling a computer chip company and asking them to fabricate a custom processor for general business needs. What about a server? Unlikely, but perhaps. What about your core business applications, like finance, human resources, or customer relationship management (CRM)? Somewhat more likely. How about applications that provide capabilities that differentiate you in the marketplace? OK, now we’re talking.

In other words, virtually no enterprise has any rational motivation to specify custom infrastructure. Today’s Infrastructure-as-a-Service (IaaS) will do, especially considering how many configuration choices are available today: processor speed, operating system (as long as you want Windows or a flavor of Linux), memory, storage, and network are all user configurable and provisionable. Furthermore, there’s no reason to customize your dev, test, or deployment environments, so might as well use a Platform-as-a-Service (PaaS) offering.

But what about the applications? For non-strategic apps like CRM, might as well use Software-as-a-Service (SaaS) like Salesforce. No executives in their right mind would say that their customer relationship needs are so unique that they should code their own CRM system. So, what about those strategic apps, the ones  that offer our differentiated capabilities or information to our customers? If an existing SaaS app won’t do, well, that’s what PaaS and IaaS are for: building and hosting our custom apps for us, respectively.

Still not convinced? Consider the competitive risk: the risk of spending too much money on unnecessary capabilities. While your competition is leveraging the Cloud, focusing their efforts on their true strategic differentiation in the market and saving buckets of dough everywhere else, you’re busy pouring cash into building yet another widget that might as well be the same widget you can get much more cheaply in the Cloud. If doing something unique and different doesn’t help the bottom line, then you’re simply wasting money. The asteroid is almost here. Which would you rather be, a dinosaur or a mammal?

The ZapThink Take

Outsourcing commodity capabilities to the low-cost provider while focusing your strategic value-add on customized offerings is an oft-repeated pattern in the world of business, but it hasn’t really taken hold in the world of IT until the rise of Cloud Computing. The reason it’s taken so long for the techies is because we’ve never been able to separate control and responsibility in the past as well as we can today. Before the Cloud, if we wanted to outsource one, then the other went along for the ride. Any enterprise that outsourced their entire IT operation went down this road. Sure, your technology becomes somebody else’s responsibility, but you end up giving up control as well.

Perhaps the greatest challenge with maintaining such control with the Cloud is that it raises the stakes on governance, leading to what we call next-generation governance in our ZapThink 2020 Poster as well as my new book, The Agile Architecture Revolution. The Cloud’s automated self-service represents powerful tools in the hands of people across our organization. Without a proactive, automated approach to governance, we risk running off the rails. Such issues are endemic in today’s technology environments: from Bring-Your-Own-Device (BYOD) challenges to SOA governance to rogue Clouds, we must learn how to maintain control while maintaining the agility benefit such powerful technology dangles in front of us. But until we learn to delegate responsibility for the underlying technology to Public Cloud Providers, we’ll never be able to maintain control cost-effectively while maintaining our competitiveness.

Image source: Diego David Garcia

At last week’s CloudConnect conference in Santa Clara, California, a speaker asked the audience how many people were implementing Private Clouds. A few dozen of the fifty or so attendees raised their hands. Then he asked how many of them were implementing automated self-service. All the hands went down.

Now, we can argue that because automated self-service is an essential Cloud characteristic, nobody in the room was in fact implementing Private Cloud at all. But take a closer look, and the lack of emphasis on self-service Private Clouds is a telling indicator of the state of Cloud Computing (in particular, Infrastructure-as-a-Service, or IaaS) in the enterprise. If an enterprise IT shop were to truly implement a self-service Private Cloud, and actually got it to work properly, then the enterprise development teams would be able to manage the entire production environment for themselves. There’d be nothing left for the operational IT folks to do except make sure to replace bad hard drives and the like. No more server or network administration. No more break/fix. No more reason to get that healthy salary – or any salary at all, for that matter.

That’s the fear (often unspoken) of many an IT professional. Cloud will take our jobs! And not only that, giving the development team responsibility for managing the operational environment masquerading as IaaS is a recipe for disaster. It’s no wonder nobody in the aforementioned conference session admitted to implementing automated self-service. After all, automated self-service turns the Cloud into the Devil’s playground.

The Rise of Full Lifecycle Governance

It may seem that Cloud is playing the villain in this melodrama, but in fact, such challenges predate the Cloud by a decade or more. As we have long discussed in our Licensed ZapThink Architect (LZA) SOA course, the move to SOA requires full lifecycle governance. IT shops that divide their governance activities into separate app/dev and operations groups, or worse, have no governance at all, are ill-prepared to implement SOA, because with SOA, the fun begins after deployment of Services. The conclusion of the development phase, in theory, brings the publication of Services. Consumption, composition, and versioning of those Services takes place subsequently, now that Services are the responsibility of operations. Unless the IT shop coordinates their SOA policies across the lifecycle, expect no end of problems as the app dev team tries to monkey with production software.

Today, add Cloud to the mix. The rise of Cloud in the enterprise adds an entirely new dimension to the requirement for full-lifecycle governance, because we’re not just reinventing how to consume and compose application functionality and data as we did with SOA, we’re revamping the entire operational environment. IT will never be the same again. But in spite of the doom and gloom pronouncements of many an old-guard admin, the Cloud doesn’t put ops folks out of a job. It does, however, redefine their role.

Enter DevOps

The idea behind DevOps is to take the concept of full lifecycle governance and bring it down to the project level. Instead of the app dev team chucking code over the wall to ops, bring the ops folks together with the developers and testers so that code iterations can include the operational phases of the lifecycle as well. In essence, DevOps extends the principles of the Agile Manifesto – working with stakeholders to focus on delivering working software that meets changing business needs – to include running the software, not just building it.

Sounds good, but the multifaceted challenges facing successful DevOps are personal, technical, architectural, as well as organizational. On the personal level, ops personnel must change their working situation, often moving their desk and dealing with different people, learning different technologies, and following different processes. On the technical level, DevOps requires continuous integration, continuous testing, and automated deployment capabilities that even today’s more advanced Platform-as-a-Service (PaaS) offerings are only now in the process of rolling out. Next, layer on architectural considerations, including how well the existing code and integration environments support policy-driven automation, which is the essence of Agile Architecture that I discuss in my new book, The Agile Architecture Revolution. But the most significant change that DevOps introduces to the IT shop, even more significant than architectural issues, are the necessary organizational changes.

Typically, app dev and operations report up to the CIO through different managers, say a VP of development or engineering plus a VP of operations. This traditional organizational structure doesn’t make sense any more. Instead, there should be a VP of software programs or portfolios, where teams of developers, testers, as well as ops people report up through the single VP. However, even this simplified org chart doesn’t tell the whole story for most enterprise IT shops, because the focus isn’t entirely on software development. It’s also on integration. And as such shops move to the Cloud, the challenge then becomes how to implement and manage a Hybrid Cloud-based environment.

The Enterprise DevOps Challenge: Hybrid Clouds

As we discuss in our Cloud Computing for Architects and Enterprise Cloud Computing courses, it’s important to place the Cloud into the enterprise context. In other words, all that heterogeneous legacy you’ve been struggling with for years. Sure, it might sound good to the executives to simply move all that old code to the Cloud, but in most situations, such migration is impractical or simply impossible. Instead, some capabilities should remain on-premise while others will do just fine in the Cloud. Now the challenge is connecting them together.

Enter the Hybrid Cloud. In reality, there are many different types of Hybrid Clouds: on premise to Public Cloud, on premise to Private Cloud, Private Cloud to Public Cloud, and every other combination you can think of. Furthermore, most enterprise mobile development falls into the broad Hybrid Cloud category, with the rise of Mobile-Backend-as-a-Service or MBaaS (more about MBaaS in a future ZapFlash). Face it, the story of today’s technology is one of connecting things, rather than running apps in isolation. The Cloud multiplies the number of opportunities for establishing such connections.

This inherent complexity endemic to virtually all enterprise IT shops complicates the DevOps story. Instead of simply focusing on revamping development teams, now the CIO must consider on-premise vs. Cloud-based development as well as on-premise vs. Cloud-based deployment, and then how to integrate the whole shebang. In some cases, IT shops will have traditional on-premise development teams chucking code over to on-premise deployment teams while at the same time building Cloud-based development/deployment teams that follow the DevOps model. In other cases, some development will take place on PaaS in the Cloud for deployment on-premise, or conceivably some development will be on-premise for deployment on IaaS. If you’re confused at this point, you’re not alone.

Depending on the types of development and integration challenges your shop faces, therefore, you may find a different org chart to be in order. For example, you may have a traditional on-premise app dev division coupled with a traditional ops division, now supplemented with a DevOps-based Cloud portfolio division. Or if your organization is able to bring DevOps to on-premise development and deployment, then you might have an on-premise DevOps division to go along with the Cloud-based one. The bottom line, however, is that all these organizational models are as yet unproven. Only time will tell how many times we’ll need to shake up the IT org chart before the dust finally settles.

The ZapThink Take

Over a year ago, we pointed out that we were entering DevOps’ “golden age.” The automated self-service capabilities of the Cloud (in particular, Public Clouds) are driving organizations to rethink how they handle operations, while at the same time empowering developers and testers to provision IT capabilities for themselves. In the enterprise IT context, however, the story is necessarily murkier, as there are so many moving parts to existing legacy environments.

One important question remains: will DevOps gain traction in traditional, on-premise development/deployment environments independent of the move to the Cloud? Or will such environments remain stuck in the IT governance dark ages until such time as anything and everything moves to the Cloud? The answer to this question circles back to the personal considerations of the individuals involved. Which is better, to resist change when change isn’t mandatory, or to take a page out of the Cloud’s developing organizational playbook to shake up traditional IT, even before you move to the Cloud? To answer a question with yet another question: is the current way of doing things working for you? If not, then you may want to consider the move to DevOps independently of your decision if and when to move to the Cloud.

Image credit: Hobbies on a Budget

There’s nothing worse than opening your water bill and finding that it’s a hundred dollars more than you expected. You scour your house and find the culprit: a leaky toilet or perhaps a dripping faucet. Hard to believe a simple drip drip drip can run up your water bill so dramatically, but those drips add up, quickly.

Replace the water with IT capability, and you have Cloud Computing. The pay-as-you-go utility model for Cloud promises dramatic cost savings, especially when unpredictable demand in a traditional, on-premise environment would require poorly utilized servers, sitting mostly idle on the off chance some spike in demand comes along. But just as with your water bill, there are many ways for your Cloud bill to go through the roof unexpectedly. Recognizing the Cloud equivalents to your problem plumbing fixtures can make the difference between saving money in the Cloud and flushing your savings down the drain.

Listening for the Drip

In fact, there are so many different mistakes you can make that will run up your Cloud bill unnecessarily that it’s a wonder anybody can save money in the Cloud at all. The most commonly discussed of these mistakes is the problem of zombie instances. If your IT shop doesn’t have adequate deprovisioning policies, then people will tend to leave instances running long after they’ve served their purpose. Over time people forget why they’re still around, and nobody will want to deprovision them on the off chance there’s something important on them.

But zombie instances are just the precipice of the Cloud money pit. An entire class of coding mistakes can also inadvertently consume Cloud resources. For example, a simple infinite loop may not only cause an application to hang, it may also spin the Cloud meter as well. And while such a loop in a single piece of code is a beginner’s mistake, sometimes the loops involve multiple instances, for example, when two apps call each other ad infinitum. Other basic coding errors that can run up the bill include memory leaks or pushing garbage collection to an excessively low priority. In a traditional deployment environment such mistakes may simply cause a process to freeze once the memory runs out, but the Cloud may provision additional instances to deal with the low memory situation. Cha-ching!

Then there are the attacks from outside. Since the ZapThink Web site runs on WordPress, we are always subject to floods of comment spam, which we must filter out. If we were running in the Cloud, then we’d have to pay for the privilege, adding insult to injury. And what about Denial of Service (DoS) attacks? Such attacks can consume immense amounts of bandwidth and other pay-as-you-go resources, again costing you more. If you’ve configured the Cloud to autoscale to deal with spikes in demand, then there may be no limit as to how much damage a DoS attack can do to your monthly Cloud bill.

Improperly configured dependencies among different workloads (or parts of a workload) can also lead to unnecessary expense. For example, let’s say you provision three Web servers and three app servers in the Cloud, and set up a round-robin policy where any of the Web servers can talk to any of the app servers. So far so good, but what happens when the whole shebang scales up? Instead of nine possible communication paths, you quickly have hundreds, consuming pay-as-you-go Cloud networking resources and also possibly bogging down your servers. And what does the Cloud do when your servers bog down? Provision more servers of course, making the problem worse.

This network communication problem can be especially onerous when you’re provisioning a Hadoop cluster in the Cloud. With Hadoop, your analytics algorithm runs in parallel on many different nodes, and the Cloud may increase the number of nodes depending on how big your Big Data sets are. As a result, if you construct the algorithm so that each node must talk to every other node, you have the same kind of network explosion issue detailed above. Even if the nodes aren’t talking to each other, but instead the algorithm “phones home” to a system outside the Hadoop cluster (say, a simple query to an on-premise system of record), then not only will you have a spike in network traffic in the Cloud, but you’ve also mounted your very own DoS attack on your mission-critical system. Your admin will not be at all pleased with you after that kind of mistake.

Another Hadoop configuration problem is actually an example of a broader issue that might run up your bill even if Hadoop isn’t on your to-do list: how you set up mirroring. By default, the Hadoop File System (HDFS) calls for three instances of each data node, scattered redundantly onto different VM instances similar to how RAID drives work. As a result, many instances can fail at random without hosing your Hadoop job. The downside to this mirroring, of course, is that it triples the size of your data sets, which triples your data storage bill for the time your HDFS is provisioned. You can configure the tripling factor, but it’s not a good idea to set it any lower. Setting it higher, of course, sends your costs into the stratosphere.

Of course, you want to do mirroring in the Cloud in other situations as well, for example, whenever you want to set up a disaster recovery or failover scenario – or simply when you want your data to have higher availability than the Cloud would normally offer. But beware: excessive mirroring is a great way to balloon your Cloud cost, especially if you’re dealing with large quantities of data. It’s also important to remember that mirroring may already be built into your Cloud storage offering. Don’t layer on additional mirroring if what you get out of the box is sufficient to meet your needs.

Sometimes the problem has to do with how you set up your Cloud instances ahead of time. Developers are used to requesting all the server resources they will require at design time, in order to avoid problems down the road. But that way of thinking is obsolete in the Cloud, since it leads to the provisioning of underutilized resources. Instead, developers should provision the minimum necessary resources for going live, and let the Cloud scale up if – and when – their workload needs additional resources.

Setting up autoscaling inefficiently is a related, but different problem. Let’s say you establish an autoscaling threshold of 80%: if an instance reaches 80% utilization, then the Cloud provisions another instance. Simple enough, but why 80%? If 90% will meet your elasticity requirements just as well, then the 80% configuration will pour your money down the drain. Unfortunately, there’s no hard and fast rule about how to set the perfect autoscaling threshold, since it depends on just how spiky your traffic spikes are, how long the Cloud will take to provision additional instances, and other factors. In other words, expect to tweak this number over time to squeeze the most cost savings out of your Cloud.

The ZapThink Take

In most of the developed world, drinking water is cheap, plentiful, and ubiquitous. In the developing world, however, it’s a different story. For those communities where drinking water is a precious resource, people have entirely different water consumption habits than we do here in the US, for example.

In the Cloud, there is no developed world. We’re all in that remote village, where only recently did a single well replace daily five-hour treks to a polluted river. Yes, we have water on tap, thank goodness – but we cannot fool ourselves that it is so plentiful and inexpensive that we can leave the faucet running while we brush our teeth.

Perhaps the most important lesson of this ZapFlash, however, is that the responsibility for proper consumption of Cloud resources doesn’t fall to a single role in our organization. Rather, developers, operations personnel, as well as the managers responsible for the Cloud provider business relationship must work together to ensure the code is correct, the configurations are efficient, and the costs are transparent and carefully monitored. True, squeezing value out of the Cloud is more difficult than we first thought. But remember, we’re still in the early days. Give it a few years, and we won’t have to think twice about simply turning a faucet to get all the Cloud we want.

Image credit: Maegan Tintari

Combine the supercharged Cloud Computing marketplace with the ubergeek cred of the open source movement, and you’re bound to have some Mentos-in-Diet-Coke moments. Such is the case with today’s Cloud Service Orchestration (CSO) platforms. At this moment in time, the leading CSO platform is OpenStack. Dozens of vendors and Cloud Service Providers (CSPs) have piled on this effort, from Rackspace to HP to Dell, and most recently, IBM has announced that they’re going all in as well. Fizzy to be sure, but all Coke, no Mentos.

Then there are CloudStack, Eucalyptus, and a few other OpenStack competitors. With all the momentum of OpenStack, it might seem that these open source alternatives are little more than also-rans, doomed to drop further and further behind the burgeoning leader. But there’s more to this story. This is no techie my-open-source-is-better-than-your-open-source battle of principle, of interest only to the cognoscenti. On the contrary: big players are now involved, and they’re placing increasingly large bets. Add a good healthy dose of Mentos – only this time, the Mentos are money.

Understanding the CSO Marketplace

Look around the Infrastructure-as-a-Service (IaaS) market. Notice that elephant in the corner? That’s Amazon Web Services (AWS). The IaaS market simply doesn’t make sense unless you realize that AWS essentially invented IaaS. And by invented, we mean actually got it to work. Which if you think about it, is rather atypical for most technology vendors. Your average software vendor will identify a new market opportunity, take some old stuff they’ve been struggling to sell, give it a nice new coat of PowerPoint, and shoehorn it into the new market. If customers bite, then the vendor will devote resources into making the product actually do what it’s supposed to do. Eventually. We hope.

But AWS is different. Amazon.com is an online reseller, not a software vendor. They think more like Wal-Mart than IBM. They figured out elasticity at scale, added customer self-service, and christened it IaaS. Then they grew it exponentially, defining what Cloud Computing really means. Today, they leverage their market dominance and economies of scale to continually lower prices, squeezing their competitors’ margins to nothing. It worked for Rockefeller’s Standard Oil, and it works for Wal-Mart. Now it’s working for Amazon.

But as with any market, there are always competitors looking to carve off a bit of opportunity for themselves. Given AWS’s dominance, however, there are two basic approaches to competing with Amazon: do what AWS is doing but try to do it a bit better (say, with Rackspace’s promise of better customer service), or do something similar to AWS but different enough to interest some segment of the market (leading in particular to the Enterprise Public Cloud space populated by the likes of Verizon Terremark and Savvis, to name a few).

And then there are the big vendors like HP and IBM, who not only offer a range of enterprise software products, but who also offer enterprise data center managed services and associated consulting. Such vendors want to play two sides of this market: they want to be Public Cloud providers in their own right, and also offer “turnkey” Cloud gear to customers who want to build their own Private Clouds. Enter OpenStack. Both of the aforementioned vendors as well as the smaller players realize that piecing together their own Cloud offerings will never enable them to catch up to AWS. Instead, they’re joining forces to build out a common Cloud infrastructure platform that supports the primary capabilities of IaaS (compute, storage, database, and network), as well as providing the infrastructure platform for Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS) capabilities down the road. The open source model is perfect for such collaboration, as the Apache license allows contributors to take the shared codebase and build out whatever proprietary add-ons they like.

Perhaps the most touted, and yet most challenging benefits of the promised all-OpenStack world is the holy grail of workload portability. In theory, if you’re running your workloads on one OpenStack-based Cloud, you should be able to move them lock stock and barrel to any other OpenStack-based Cloud, even if it belongs to a different CSP. Workload portability is the key to Cloud-based failover and disaster recovery, Cloud bursting, and multi-Cloud deployments. Today, workload portability requires a single proprietary platform, and only VMware offers such portability. AWS offers a measure of portability within its Cloud, but will face challenges supporting portability between itself and other providers. As a result, if OpenStack can get portability to work properly, participating CSPs will have a competitive lever against Amazon.

Achieving a strong competitive position against AWS with OpenStack is easier said than done, however. OpenStack is a work in progress, and many bits and pieces are still missing. Open source efforts take time to mature, and meanwhile, AWS keeps growing. In response, the players in this space are taking different tacks to build mature offerings that have a hope of carving off a viable chunk of the IaaS marketplace:

• Rackspace is trying to capitalize on its OpenStack leadership position and the aforementioned customer service to provide a viable alternative to AWS. They are also touting the workload portability benefits of OpenStack. But downward pricing pressure combined with the holes in OpenStack capabilities are pounding on Rackspace’s stock price.
• Faced with the demise of its traditional PC business, Dell is focusing on its Boomi B2B integration product, recently rechristened as Cloud integration. Cloud integration is a critical enabler of Hybrid Clouds, but doesn’t address the workload portability challenge. As a result, Dell’s Cloud marketing efforts are focused on the benefits of integration over portability. Dell’s recent acquisition of Quest Software also hints at a Microsoft application migration strategy for Dell Cloud.
• HP wants to rush its Enterprise Public Cloud offering to market, and it doesn’t want to wait for OpenStack to mature. Instead, it’s hammering out its own version of OpenStack, essentially forking the OpenStack codebase to its own ends, according to Nnamdi Orakwue, vice president for Dell Cloud. Such a move may pay off for HP, but increases the risk that the HP add-ons to OpenStack will have quality issues.
• IBM recently announced that they are “all in” with OpenStack with the rollout of  IBM SmartCloud Orchestrator built on the platform.  But IBM has a problem: the rest of their SmartCloud suite isn’t built on OpenStack, leaving them to scramble to rewrite a number of existing products leveraging OpenStack’s incomplete codebase, while in the meantime, integrating the mishmash of SmartCloud components at the PowerPoint layer.
• Red Hat is making good progress hammering out what they consider an “enterprise” deployment of OpenStack. As perhaps the leading enterprise open source vendor, they are well-positioned to lead this segment of the market, but it still remains to be seen whether enterprise customers will want to  build all open source Private Clouds in the near term, as the products gradually mature. On the other hand, IBM has a history of leveraging Red Hat’s open source products, so an IBM/Red Hat partnership may move SmartCloud forward more quickly than IBM might be able to accomplish on its own.

CSO Wild Card: CloudStack

There are several more players in this story, but one more warrants a discussion: Citrix. The desktop virtualization leader had been one face in the OpenStack crowd, but they suddenly decided to switch horses and take a contrarian strategy. They ditched OpenStack, took their 2011 Cloud.com acquisition and donated the code to CloudStack. Then they switched CloudStack’s licensing model from GNU (derivative products must be licensed under GNU) to Apache (OK to build proprietary offerings on top of the open source codebase), and subsequently passed the entire CloudStack effort along to the Apache Foundation, where it’s now in incubation.

There are far fewer players on the CloudStack team than OpenStack’s, and its core value proposition is quite similar to OpenStack, so on first glance, Citrix’s move raises eyebrows. After all, why bail on the market leader to join the underdog? But look more closely, and it seems that Citrix may be onto something.

First, Citrix’s open source Cloud strategy is not all about CloudStack. They’re also heavily invested in Xen. Xen is one of the two leading open source virtualization platforms, and provides the underpinnings to many commercial virtualization products on the market today. Citrix’s 2007 acquisition of XenSource positioned them as a Xen leader, and they’ve been driving development of the Xen codebase ever since.

Citrix’s heavy investment in Xen bucks the conventional virtualization wisdom: since Xen’s primary competitor, KVM (Kernel-based Virtual Machine) is distributed as part of standard Linux distros, KVM is the no-brainer choice for the virtualization component of open source CSOs. After all, it’s essentially part of Linux, so any CSP (save those focusing on Windows-centric IaaS) don’t have to lift a finger to build their offerings on KVM. Citrix, however, picked up on a critical fact: KVM is simply not as good as Xen. And now that Citrix has been pushing Xen to mature for half a dozen years, Xen is a far better choice for building turnkey Cloud solutions than KVM. So they Citrix combined Xen and CloudStack into a single Cloud architecture they dubbed Windsor, which forms the basis of their CloudPlatform offering.

And therein lies the key to Citrix’s contrarian strategy: CloudPlatform is a turnkey Cloud solution for customers who want to deploy Private Clouds – or as close to turnkey as today’s still nascent Cloud market can offer. Citrix is passing on the opportunity to be their own CSP (at least for now), instead focusing on driving CloudStack and Xen maturity to the point that they can put together a complete Cloud infrastructure software offering. In other words, they are focusing on a niche and giving it all they got.

The ZapThink Take

If this ZapFlash makes comprehending the IaaS marketplace look like herding cats, you’re right. AWS has gotten so big, so fast, and their products are so good, that everyone else is scrambling to put something together that will carve off a piece of what promises to be an immense market. But customers are holding the cards, because everyone knows how AWS works, which means that everyone knows how IaaS is supposed to work. If a vendor or CSP brings an offering to market that doesn’t compare with AWS on quality, functionality, or cost, then customers will steer clear, no matter how good the contenders’ PowerPoints are.

But as with feline wrangling, it’s anybody’s guess where this tabby or that calico is heading next. If anyone truly challenges Amazon’s dominance, who will it be? Rackspace? IBM? Dell? Or any of the dozens of other four-legged critters just looking for a warm spot in the sun? And then there’s the turnkey Cloud solution angle. Today, building out your own Private Cloud is difficult, expensive, and fraught with peril. But if tomorrow brings simple, low cost, low risk Private Clouds to the enterprise, how will that impact the Public CSP marketplace? You pays your money, you takes your chances. But today, the safe IaaS choice is AWS, unless you have a really good reason for selecting an alternative.

Scenario #1: out of the blue, your boss calls, looking for some long-forgotten entry in a spreadsheet from 1989. Where do you look? Or consider scenario #2: said boss calls again, only this time she wants you to analyze customer purchasing behavior…going back to 1980. Similar problem, only instead of finding a single datum, you must find years of ancient information and prepare it for analysis with a modern business intelligence tool.

The answer, of course, is archiving. Fortunately, you (or your predecessor, or predecessor’s predecessor) have been archiving important—or potentially important—corporate data since your organization first started using computers back in the 1960s. So all you have to do to keep your boss happy is find the appropriate archives, recover the necessary data, and you’re good to go, right?

Not so fast. There are a number of gotchas to this story, some more obvious than others. Cloud to the rescue? Perhaps, but many archiving challenges remain, and the Cloud actually introduces some new speed bumps as well. Now factor in Big Data. Sure, Big Data are big, so archiving Big Data requires a big archive. Lucky you—vendors have already been knocking on your door peddling Big Data archiving solutions. Now can you finally breathe easy? Maybe, maybe not. Here’s why.

Archiving: The Long View

So much of our digital lives have taken place over the last twenty years or so that we forget that digital computing dates back to the 1940s—and furthermore, we forget that this sixty-odd year lifetime of the Information Age is really only the first act of perhaps centuries of computing before humankind either evolves past zeroes and ones altogether or kills itself off in the process. Our technologies for archiving information, however, are woefully shortsighted, for several reasons:

• Hardware obsolescence (three to five years) – Using a hard drive or tape drive for archiving? It won’t be long till the hardware is obsolete. You may get more life out of the gear you own, but one it wears out, you’ll be stuck. Anyone who archived to laser disk in the 1980s has been down this road.

• File format obsolescence (five to ten years) – True, today’s Office products can probably read that file originally saved in the Microsoft Excel version 1 file format back in the day, but what about those VisiCalc or Lotus 123 files? Tools that will convert such files to their modern equivalents will eventually grow increasingly scarce, and you always risk the possibility that they won’t handle the conversion properly, leading to data corruption. If your data are encrypted, then your encryption format falls into the file format obsolescence bucket as well. And what about the programs themselves? From simple spreadsheet formulas to complex legacy spaghetti code, how do you archive algorithms in an obsolescence-proof format?

• Media obsolescence (ten to fifteen years) – CD-ROMs and digital backup tapes have an expected lifetime. Keeping them cool and dry can extend their life, but actually using them will shorten it. Do you really want to rely upon a fifteen-year-old backup tape for critical information?

• Computing paradigm obsolescence (fifty years perhaps; it’s anybody’s guess) – will quantum computing or biological processors or some other futuristic gear drive binary digital technologies into the Stone Age? Only time will tell. But if you are forward thinking enough to archive information for the 22^nd century, there’s no telling what you’ll need to do to maintain the viability of your archives in a post-binary world.

Cloud to the Rescue?

On the surface, letting your Cloud Service Provider (CSP) archive your data solves many of these issues. Not only are the new archiving services like Amazon Glacier impressively cost-effective, but we can feel reasonably comfortable counting on today’s CSPs to migrate our data from one hardware/media platform to the next over time as technology advances. So, can Cloud solve all your archiving issues?

At some point the answer may be yes, but Cloud Computing is still far too immature to jump to such a conclusion. Will your CSP still be in business decades from now? As the CSP market undergoes its inevitable consolidation phase, will the new CSP who bought out your old CSP handle your archive properly? Only time will tell.

But even if the CSPs rise to the archiving challenge, you may still have the file format challenge. Sure, archiving those old Lotus 123 files in the Cloud is a piece of cake, but that doesn’t mean that your CSP will return them in Excel version 21.3 format ten years hence—an unfortunate and unintentional example of garbage in the Cloud.

The Big Data Old Tail

You might think that the challenges inherent in archiving Big Data are simply a matter of degree: bigger storage for bigger data sets, right? But thinking of Big Data as little more than extra-large data sets misses the big picture of the importance of Big Data.

The point to Big Data is that the indicated data sets continue to grow in size on an ongoing basis, continually pushing the limits of existing technology. The more capacity available for storage and processing, the larger the data sets we end up with. In other words, Big Data are by definition a moving target.

One familiar estimate states that the quantity of data in the world doubles every two years. Your organization’s Big Data may grow somewhat faster or slower than this convenient benchmark, but in any case, the point is that Big Data growth is exponential. So, taking the two-year doubling factor as a rule of thumb, we can safely say that at any point in time, half of your Big Data are less than two years old, while the other half of your Big Data are more than two years old. And of course, this ZapFlash is concerned with the older half.

The Big Data archiving challenge, therefore, is breaking down the more-than-two-years-old Big Data sets. Remember that this two-year window is true at any point in time. Thinking about the problem mathematically, then, you can conclude that a quarter of your Big Data are more than four years old, an eighth are more than six years old, etc.

Combine this math with the lesson of the first part of this ZapFlash, and a critical point emerges: byte for byte, the cost of maintaining usable archives increases the older those archives become. And yet, the relative size of those archives is vanishingly small relative to today’s and tomorrow’s Big Data. Furthermore, this problem will only get worse over time, because the size of the Old Tail continues to grow exponentially.

We call this Big Data archiving problem the Big Data Old Tail. Similar to the Long Tail argument, which focuses on the value inherent in summing up the Long Tail of customer demand for niche products, the Big Data Old Tail focuses on the costs inherent in maintaining archives of increasingly small, yet increasingly costly data as we struggle to deal with older and older information. True, perhaps the fact that the Old Tail data sets from a particular time period are small will compensate for the fact that they are costly to archive, but remember that the Old Tail continues to grow over time. Unless we deal with the Old Tail, it threatens to overwhelm us.

The ZapThink Take

The obvious question that comes to mind is whether we need to save all those old data sets anyway. After all, who cares about, say, purchasing data from 1982? And of course, you may have a business reason for deleting old information. Since information you preserve may be subject to lawsuits or other unpleasantness, you may wish to delete data once it’s legal to do so.

Fair enough. But there are perhaps far more examples of Big Data sets that your organization will wish to preserve indefinitely than data sets you’re happy to delete. From scientific data to information on market behavior to social trends, the richness of our Big Data do not simply depend on the information from the last year or two or even ten. After all, if we forget the mistakes of the past then we are doomed to repeat them. Crunching today’s Big Data can give us business intelligence, but only by crunching yesterday’s Big Data as well can we ever expect to glean wisdom from our information.

Our ZapThink 2020 poster lays out our complex web of predictions for enterprise IT in the year 2020. You might think that semantic operability is an important part of this story; after all, several groups have been heads down working on the problem of how to teach computers to agree on the meaning of the information they exchange for years now. But look again: we relegate semantics to the lower right corner, where we point out that we don’t believe there will be much progress in this area by 2020. Eventually, maybe, but even though semantic interoperability appears to be within our grasp, it behaves more like the pot of gold under the rainbow. The closer you get to the rainbow, the farther away it appears.

What gives? ZapThink usually takes an optimistic perspective about the future of technology, but we’re decidedly pessimistic about the prospects for semantic interoperability. The problem as we see it comes down to the human understanding of language. All efforts to standardize meanings in order to facilitate semantic interoperability strip out vagueness and ambiguity from data, and presume a single, universal underlying grammar. After all, isn’t the goal to foster precise, unambiguous, and consistent communication between systems? The problem is, human communication is inherently vague, ambiguous, and relative. The way humans understand the world, the way we think, and the way we put our thoughts into language require both vagueness and ambiguity. Without them, we lose important aspects of meaning. Furthermore, how we structure our language is culturally and linguistically relative. As a result, current semantic interoperability efforts will be able to address a certain class of problems, but in the grand scheme of things, that class of problems is a relatively small subset of the types of communication we would prefer to automate between systems.

The Importance of Vagueness

Ironically, to discuss semantics we must first define our terms. A term is vague when it’s impossible to say whether the term applies in certain circumstances, for example, “my face is red.” Just how red does it have to be before we’re sure it’s red? In contrast, a term is ambiguous when it’s possible to interpret it in more than one way. For example, “I’m going to a bank” might mean that I’m going to a financial institution or to the side of a river.

Vagueness leads to knotty problems in philosophy that impact our ability to provide semantic interoperability. So, let’s go back to philosophy class, and study the sorites paradox. If you have a heap of sand and you take away a single grain, do you still have a heap of sand? Certainly. OK, repeat the process. Clearly, when you get down to a single grain of sand remaining, you no longer have a heap. So, when did the heap cease to be a heap?

Philosophers and linguists have been arguing over how to solve the sorites paradox for over a century now (yes, I know, they should find something more useful to do with their time). One answer: put your foot down and establish a precise boundary. 1,000 or more grains of sand are a heap, but 999 or less are not. Our computers will have no problem with such a resolution to the paradox, but it doesn’t accurately represent what we really mean by a heap. After all, if 1,000 grains constitutes a heap, wouldn’t 999? Central to the meaning of the term “heap” is its inherent vagueness.

Another solution: yes, there is some number of grains of sand where a heap ceases to be a heap, but we can’t know what it is. This resolution might satisfy some philosophers, but it doesn’t help our computers make sense out of our language. A third approach: instead of considering “is a heap” and “isn’t a heap” as the only two possible values, define a spectrum of intermediary values, or perhaps a continuum of values. The computer scientists are likely to be happy with this answer, as it lends itself to fuzzy logic: the statement “this pile of sand is a heap” might be, say, 40% true. Yes, we can do our fuzzy logic math now, but we’ve still lost some fundamental elements of meaning.

To bring back our natural language-based understanding of the sorites paradox, let’s step away from an overly analytical approach to the problem and try to look at the paradox from a human perspective. How, for example, would a seven-year-old describe the heap of sand as you take away a grain of sand at a time? They might answer, “well, it’s a smaller heap” or “it’s kinda a heap” or “it’s a little heap” or “it’s not really a heap,” etc. Such expressions are clearly not precise. Our computers wouldn’t be able to make much sense out of them. But these simple, even childish expressions are how people really speak and how people truly understand vagueness.

The important takeaway here is that vagueness isn’t a property relegated to heaps and blushing faces. It’s a ubiquitous property of virtually all human communication, even within the business context. Take for example an insurance policy. Insurance policies have a number of properties (policy holder, underwriter, insured property, deductible, etc.) and relationships to other business entities (policy application, underwriting documentation, claims forms, etc.) Now let’s add or take away individual properties and relationships from our canonical understanding of an insurance policy one at a time. Is it still a policy? Clearly, if we take away everything that makes a policy a policy then it’s no longer a policy. But if we take away a single property, we’re likely to say it’s still a policy. So where do we draw the line? If philosophers and linguists haven’t solved this problem in over a century, don’t expect your semantic interoperability tool to make much headway either.

The Problem of Linguistic Relativity

Another century-long battle in the world of linguistics is the fray over linguistic relativity vs. linguistic universality. Linguistic relativity is the position that language affects how speakers see their world, and by extension, how they think. In the other corner is Noam Chomsky’s universal grammar, the linguistic theory that grammar is hardwired into the brain, and hence universal across all peoples regardless of their language or their culture. Theoretical work on a universal grammar has led to dramatic advances in natural language translation, and we all get to use and appreciate Google Translate and its brethren as a result. But while Google Translate is a miraculous tool indeed (especially for us Star Trek fans who marveled at the Universal Translator), it doesn’t take a polyglot to realize that the state of the art for such technology still leaves much to be desired.

Linguistic relativity, however, goes at the heart of the semantic interoperability challenge. Take for example, one of today’s most useful semantic standards: the Resource Description Framework (RDF). RDF is a metadata data model intended for making statements about resources (in particular, Web-based resources) in the form of subject-predicate-object expressions. For example, you might be able to express the statement “ZapThink wrote this ZapFlash” in the triplet consisting of “ZapThink” (the subject); “wrote” (the predicate); and “this ZapFlash” (the object). Take this basic triplet building block and you can build semantic webs of arbitrary complexity, with the eventual goal of describing the relationships among all business entities within a particular business context.

The problem with the approach RDF takes, however, is that the subject-predicate-object structure is Eurocentric. Non-European languages (and hence, non-European speakers) don’t necessarily think in sentences that follow this structure. And furthermore, this problem isn’t new. In fact, the research into this phenomenon dates back to the 1940s, with the work of linguist Benjamin Lee Whorf. Whorf conducted linguistic research among the Hopi and other Native American peoples, and thus established an empirical basis for linguistic relativity. The illustration below comes from one of his seminal papers on the subject:

In the graphic above, Whorf compares a simple sentence, “I clean it with a ramrod,” where “it” refers to a gun, in English and Shawnee. The English sentence predictably follows the subject-predicate-object format that RDF leverages. The Shawnee translation, however, translates literally to “dry space/interior of hole/by motion of tool or instrument.” Not only is there no one-to-one correspondence between parts of speech across the two sentences, but the entire context of the expression is different. If you were in the unenviable position of establishing RDF-based semantic interoperability between, say, a British business and a Shawnee business, you’d find RDF far too culturally specific to rise to the challenge.

The ZapThink Take

We have tools for semantic interoperability today, of course – but all such tools require the human step of configuring or training the tool to understand the properties and relationships among entities. Once you’ve trained the tool, it’s possible to automate many semantic interactions. But to get this process started, we must get together in a room with the people we want to communicate with and hammer out the meanings of the terms we’d like to use.

This human component to semantic interoperability actually dates to the Stone Age. How did we do business in the Stone Age? Say your tribe was on the coast, so you had fish. You were getting tired of fish, so you and your tribemates decided to pack up some fish and bring the bundle to the next village where they had fruit. You showed up at the village market, only you had no common language. So what did you do? You held up some fish, pointed to some fruit, grunted, and waved your hands. If you established a basis of communication, you conducted business, and went home with some fruit. If not, then you went home empty handed (or you pulled out your clubs and attacked, but that’s another story). Cut to the 21^st century, and little has changed. People still have to get together and establish a basis of communication as human beings in order to facilitate semantic interoperability. But fully automating such interoperability is as close as the next rainbow.

The problem with Big Data is that, well, Big Data are big. Really big. We’re talking terabytes. Petabytes. Zettabytes. Whatever’s-even-bigger-bytes. And of course, we want to solve all our Big Data challenges in the Cloud. If only we could get those gigando-bytes into the Cloud in the first place. And there’s the rub.

Uploading Big Data from our internal network to the Cloud via an Internet connection is as practical as filling a swimming pool through a drinking straw. It doesn’t matter how sophisticated our Big Data analytics, how super-duper our Hadoopers. If we can’t efficiently get our data where we need them when we need them, we’re stuck.

Optimize the Pipe

Fortunately, the Big Data upload problem isn’t new. In fact, it’s been around for years, under the moniker Wide Area Network (WAN) Optimization. Fortunate for us because vendors have been working on WAN Optimization techniques for a while now, and now several of them are repurposing those techniques to help with the Cloud.

For example, Aryaka has been peddling WAN Optimization appliances for several years. Put one appliance in your local data center, a second in the remote data center, and proprietary technology moves data from one to the other at a rapid clip. Now that the Cloud has turned their world upside down, they are providing a distributed service at the remote end, a “mesh of network connections” better suited to the Cloud. In other words, Aryaka is building an offering similar to Content Delivery Networks (CDNs) like Akamai.

RainStor, in contrast, focuses primarily on a proprietary compression algorithm that promises to squeeze data into one fortieth their original size. Furthermore, RainStor’s compressed data remain directly accessible using standard SQL or even MapReduce on Hadoop with no storage-eating, time-consuming reinflation.

Then there’s Aspera, who’s found a sophisticated way around the limitations of the Transmission Control Protocol (TCP) itself. After all, TCP’s tiny packets and penchant for resending them are a large part of the reason uploading Big Data over the Internet runs like such a dog in the first place. To teach this dog a new trick or two, Aspera transfers use one TCP port for session initialization and control, and one User Datagram Protocol (UDP) port for data transfer.

UDP is an older, fire-and-forget protocol that doesn’t perform the retries that provide TCP’s reliability, but by combining the two protocols, FASP achieves nearly 100% error-free data throughput. In fact, FASP reaches the maximum transfer speed possible given the hardware on which you deploy it, and maintains maximum available throughput independent of network delay and packet loss. FASP also aggregates hundreds of concurrent transfers on commodity hardware, addressing the drinking straw problem in part by supporting hundreds of straws at once.

CloudOpt is also a player worth mentioning. Their JetStream technology takes a soup-to-nuts approach that combines compression and transmission protocol optimization with advanced data deduplication, SSL acceleration, and an ingenious approach to getting the most performance out of cached data. Or Attunity Cloudbeam, that touts file to Cloud upload, file to Cloud replication, and Cloud to Cloud replication. Attunity’s Managed File Transfer (MFT) incorporates a secure DMZ architecture, security policy enforcement, guaranteed and accelerated transfers, process automation, and audit capabilities across each stage of the file transfer process.

Finally, there’s Amazon Web Services (AWS) itself. Yes, most if not all of the vendors discussed above can firehose data into AWS’s various storage services. But AWS also offers a simple, if decidedly low-tech approach as well: AWS Import/Export. Simply ship your big hard drives to Amazon. They’ll hook them up, copy the data to your Simple Storage Service (S3) or other storage service, and ship the drive back when you’re done. This SneakerNet or “Forklifting” approach, believe it or not, can even be faster than some of the over-the-Internet optimizations for certain Big Data sets, even considering the time it takes to FedEx AWS your drives.

On Beyond Drinking Straws

The problem with most of the approaches above (excepting only Aspera and Amazon’s forklift) is that they make the drinking straw we’re using to fill that swimming pool better, faster, and bigger – but we’re still filling that damn pool with a straw. So what’s better than a straw? How about many straws? If any optimization technique improves a single connection to the Internet, then it stands to reason that establishing many connections to your Cloud provider in parallel would multiply your upload speed dramatically.

Fair enough, but let’s think out of the box here. A fundamental Big Data best practice is to bring your analytics to your data. The reasoning is that it’s hard to move your data but easy to move your software, so once your data are in the Cloud, you should also run your analytics there.

But this argument also works in reverse. If your data aren’t in the Cloud, then it may not make sense to move them to the Cloud simply to run your software there. Instead, bring your software to your data, even if they’re on premise.

Perish the thought, you say! We’re sold on Big Data in the Cloud. We’ve crunched the numbers and we know it’s going to save us money, provide more capabilities, and facilitate sharing information across our organization and the world. Fair enough. Here’s another twist for you.

Why are your Big Data sets outside the Cloud to begin with? Sure, you’re stuck with existing, legacy data sets wherever they happen to be today. But as a rule, those don’t constitute Big Data, or will cease to qualify as being large enough to warrant the Big Data label relatively soon. By definition, Big Data sets keep expanding exponentially, which means that you keep creating them with generations of newfangled tools.

In fact, there are already multitudinous sources for raw Big Data, as varied as the Big Data challenges organizations struggle with today. But many such sources are already in the Cloud, or could be moved to the Cloud simply. For example, clickthrough data from your Web sites. Such data come from your Web servers, which should be in the Cloud anyway. If your Big Data come from Web Servers scattered here and there in the Cloud, then moving the clickthrough data to a Big Data repository for processing can be handled in the same Cloud. No need for uploading.

What about data sources that aren’t already in the Cloud? Many Big Data streams come from instrumentation or sensors of some sort, from seismographs underground to EKGs in hospitals to UPC scanners in supermarkets. There’s no reason why such instrumentation shouldn’t pour their raw data feeds directly to the Cloud. What good is storing a week’s worth of supermarket purchasing data on premise anyway? You’ll want to store, process, manage, and analyze those data in the Cloud, so the sooner you get it there, the better.

The ZapThink Take

The only reason we have to worry about uploading Big Data to the Cloud in the first place is because our Big Data aren’t already in the Cloud. And broadly speaking, the reason they’re not already in the Cloud is because the Cloud isn’t everywhere. Instead, we think of the Cloud as being locked away in data centers, those alien, air conditioned facilities packed full of racks of high tech equipment.

That may be true today, but as ZapThink has discussed before, there’s nothing in the definition of Cloud Computing that requires Cloud resources to live in data centers. You might have a bit of the Cloud in your pocket, or on your laptop, in your car, or in your refrigerator. For now, this vision of the Internet of Things meeting the Cloud is mostly the stuff of science fiction. We’re only now figuring out what it means to have a ubiquitous global network of sensors, from the aforementioned EKGs and UPC scanners to traffic cameras to home thermostats. But the writing is on the wall. Just as we now don’t think twice about carrying supercomputers in our pockets, it’s only a matter of time until the Cloud itself is fully distributed and ubiquitous. When that happens, the question of moving Big Data to the Cloud will be moot. They will already be there.

Are you one of the vendors mentioned in this article and have a correction, or a vendor who should have been mentioned but wasn’t? Please feel free to comment below.

Image Source: US Navy

With contributions from Mark Levy

Many electrical and digital systems dominate today’s vehicles. For example, the onboard diagnostics port, OBD-II, has been mandatory for U.S. cars since 1996. More recently, some of these data have become more accessible to the driver. Cars can now send information directly to the dashboard alerting the driver of upcoming service requirements, or any warning for that matter, replacing the traditional red light on the dashboard.

In the last few years, however, we have witnessed significant “in-car” technical improvements such as a range of native applications like advanced navigation systems. Some examples: GM’s OnStar system (introduced in 1995) provides a great early example of automotive telematics, and Ford’s newer Sync System (powered by Microsoft) integrates in-car entertainment, vehicle diagnostics, and navigation capabilities.

Although significant data messaging takes place within today’s automobiles, it is rare to find cars where the majority of this information is directly accessible to the driver without external hardware, let alone connected to the Cloud. But of course, there’s a Cloud story here. With the rapid progress of in-car technology coupled with new, advanced Cloud services, it was only a matter of time before the new concept of Cars in the Cloud becomes a reality.

Driving Your Car in the Cloud

Today, the intersection between automobiles and consumer electronics is expanding. The features that are already on the market and will continue to improve at a rapid pace are all based on three elements:

• Smart car – Software is responsible for delivering functionality that in previous generations were made possible by analog devices
• Cloud – The ability to deliver services from the car manufacturer as well as from other providers automatically over the Internet
• Connectivity –High speed, reliable, secure communications between the car and various services that various providers are lining up to offer.

However, with every vendor Cloudwashing their offerings, crossing off words like “Web” from their marketing and writing in “Cloud,” it’s important to ask if Cars in the Cloud is really nothing more than connecting autos to the Web. While clearly some vendors have yet to move past Cloudwashing, the primary essential characteristic of Cloud, namely elasticity, is critical to the successful implementation of the Cars in the Cloud concept.  Since automobiles cannot afford software failures while driving, it would more than a minor inconvenience should any driver ever see a response on the screen saying “capacity exceeded, please try again later.”  In order to ensure that such problems never occur, the Cloud supporting the services must be elastic, and respond promptly to any failure.

Cloud provisioning is also a key concept to the successful implementation of Cars in the Cloud. When we acquire a new car, we should be able to follow a simple procedure to make our new auto known to the Cloud, so that all the services we purchased are immediately available in the car.  Clearly, in order for this process to work satisfactorily, the provisioning of such services must be done automatically with no human intervention.

Example: The Tesla Model S

The Tesla Model S is more than a next-generation electric automobile. Examining the infrastructure behind the Tesla Model S provides insight into the Car in the Cloud concept, and offers a glimpse of the future. This vehicle features a seventeen inch screen, completely replacing the manual buttons that you would normally see in a normal car.

With a typical car, there are so many physical buttons and switches that the manufacturer must roll out a new model year to make even the most basic change to the car’s functionality. In contrast, by moving these types of buttons to a digital interface, Tesla can completely change the configuration and functionality of their cars – without significant physical changes to the vehicle. For example, if enough Tesla Model S drivers request the digital button to open the sun roof to move to a different location, all that the development team must do is produce a new release and push it to all cars.  In other words, a Car in the Cloud is an automobile that can change over time without requiring owners to buy an upgraded model.

To accomplish this feat, Tesla’s programmed their on-board computer by customizing Google’s Android mobile operating system. Android is an open source stack that powers many popular consumer devices such as smartphones, tablets, and now – automobiles.  Tesla’s CEO has also said that the Model S will support third-party apps to extend each car’s functionality. Since a Tesla Model S is blessed with a large display and a handful of starter apps, it is clear that over time, the company will push more and more apps to the car via the Tesla Cloud.  But will there be a Tesla app marketplace? Will the commercial market be able to use the Tesla API and implement an ever-growing number of Tesla apps? Will there be a process for Tesla certify these apps prior to inclusion in the app store? Clearly there is an opportunity for an active marketplace that could set the trend for the automotive industry at large.

Connectivity, Scalability, and Security

The Tesla Model S is connected to the Tesla Cloud over a 3G link.  Each automobile regularly calls home, providing Tesla engineers with extensive information regarding the performance of each car.  In most cases, Tesla can fix any issues with your car without any involvement from the driver. Further, this process allows Tesla to listen to its customers and decide what features the manufacturer should add.  Since the entire auto is managed by software from the Tesla Cloud, it is possible to complete the development and testing in order to deliver upgrades with ease.  Of course, they still have to worry about scalability.  Clearly, it is imperative that the infrastructure behind the Tesla is capable of handling the volume of autos and data that the Cloud and the vehicles exchange.

The flip side of this ease of upgrade is security. Just how susceptible is a Tesla car to viruses and hackers?  For example, earlier this year, hackers commandeered software for remotely operating medical equipment and elevators.  Just how secure is the Tesla Cloud anyway? Only time will tell, but Tesla has already expended significant resources to ensure that compromises cannot happen, even though we never want to say that breaches are truly impossible. Yes, Tesla encrypts all data, establishes a VPN directly to the car, and signs and verifies all software packages. Furthermore, it’s only possible to upgrade an automobile from the Cloud when the car is charging (meaning, when it’s not moving). The upgrade takes place overnight and the next morning, you can see some few features in your car.

The ZapThink Take

Tesla’s Car in the Cloud concept still seems quite futuristic, but it’s still early days. As the technology matures, connected vehicles have the potential to change many aspects of how people own and operate their motor vehicles. Some ideas:

• Car Insurance – Currently, car insurance works much like health insurance. However, while we need of health insurance 24/7, doesn’t it make sense that we would only need the majority of my car insurance while we’re driving? Be on the lookout for “pay-as-you-drive” insurance programs in the future.
• Environmental Applications – The way that you drive has implications for your fuel range (or in the case of Tesla, the car’s battery range). Over time, cars will be able to identify opportunities to minimize fuel consumption based on driving style.
• Highway Infrastructure Applications – Yes, we have E-ZPass now, but we could greatly simplify electronic toll collection and congestion charging without having to put up more tollbooths and cameras. The amount of money that we could save for highway infrastructure is enormous.
• Convenience Applications – Connect my phone to my car and begin heating up my car when I start walking toward it. Tell me exactly where my car is at a parking lot without having to take a picture of a sign at the airport or hold up my keychain as I conduct a random walk through the lot.
• Advanced Safety and Security – I can track my iPhone if it’s stolen, and even shut it down so that anyone who takes it will find themselves with nothing more than a shiny piece of metal. My car should be able to do the same. LoJack in the Cloud.
• Automated driving – even though we’ve all heard of Google’s self-driving car, this capability still seems to be the stuff of science fiction. When it becomes a reality, however, you can bet it will take full advantage of advanced Car in the Cloud capabilities.
• Social Networking – Based on the fact that there could be roughly 300 million cars in the U.S. and more than 1 billion worldwide, we can project an opportunity to integrate social networking into cars moving forward.  For example, cars could start talking to each other in order to identify road conditions along the route. Music recommendations, rest stop data, and tourist information can stream directly into the vehicle. Your car might even start tweeting. Imagine the possibilities!

About Dov Levy

Mr. Levy is the President and CTO for Dovel Technologies, Inc., the company he co-founded. He has responsible for creating innovative solutions for the federal government by Dovel.

As an entrepreneur and a technologist with over 25 years of hands-on experience, Mr. Levy has a deep understanding of what it takes to deliver a state-of-the art solution with technologies such as Service Oriented Architecture, Cloud and Mobile computing. He has solid experience with delivering solutions using open source components.

About Mark Levy

Mark Levy is the founder of App Champs LLC, a company focused on developing and executing innovative technology strategies for small businesses. Mark has deep experience in implementing inbound marketing, mobile, and e-commerce solutions across a wide range of industries.