Republicans and other cultural conservatives feared the day the Internet at large and all its nodes would simultaneously collapse… we haven’t seen that happen yet, but we have seen industry collaborators get on the same page politically and take a stand against the war on civil liberties in the form of PIPA and SOPA. Who is supporting this terrible attempt at law-making? Those who don’t know that Intellectual property expires if you don’t maintain or share it, and some folks are trying to legislate their way out of that fact. The rest of us are taking a stand for what we believe is true. Information wants to be free, if you let it… and when you put a price on it, you’re essentially putting a price on your own head. The price of a troll’s head, technically.

@MCHAMMER says STOP! …. HAMMATIME

It’s no longer a secret that Elastic Provisioner, clients, partners, et. al. unanimously oppose legislation which attempts to control that which cannot be controlled… at the expense of the civil liberties which USA and other countries fought so hard for. In “america” we thought we were going to win some kind of war against drugs, but we were on them.. so we made peace with them. Now the war seems to be against online piracy. The same problem exists as before… people want drugs and pirated software, and those people are citizens… so it’s going to be very hard to control their behaviours. In fact, if we’re going to control them we’re all going to have to give up our privacy and innovation… just a little. Is it worth it? not really. The fact is that some people would prefer to make the Internet back into tubes again. I have decided to take a stand against it, and you’ll probably hear about it on twitter if you’re tuned into #STOPSOPA or #NOPIPASF or if you are attending The Silicon Valley PIPA and SOPA protest, brought to you by Hackers and Founders. If you’re taking computing personally, join us!

- Asher Bond

@PPLvPIPA: What sites are taking a stand against online erosion of civil liberties in the form of PIPA and SOPA?

Google, Inc.

WikiMedia / Wikipedia

Jimmy Wales has annouced that Wikipedia is abstaining from normal behaviour on this day of activism… unfortunately not in those words, but his were probably better. What I love about their implementation of the blackout is that their site appears to work normally until you submit a search query or click a topic, then all pages redirect to the blackout page.

The opposing political factions attack our Twitter account

This is from Tony Baldwin (one of my Diaspora friends).

Dog-fooding the Supercomputer

But honestly, localized computation is only half of the fun of cloud VDI. We really wanted to rock the portable UX over the internet globally. And that’s doable with a McDevOps microcloud account (contact me if you want an invite), whether or not you roll your own microcloud. Microcloud accounts will be free for engineers, developers, designers, and devops culturists… and in general free for anyone looking for work or something to hack on. But it’s not just a SaaS model, it’s a PaaS model from a software perspective. From the hardware perspective it’s a gateway appliance taking you through the pearly gates to supercomputing heaven in the cloud. Desktops are a heavy workload in and of themselves, especially in the aggregate. The problem with all the cloud hype in consumer electronics or “personal cloud” is that they’ve gotten away from cloud computing’s future value. The future value of cloud computing is that it offers scalability. As Dave Nielsen says Cloud computing is OSSM (On Demand, Scalable, Self-serviceable, and Measureable), and I say it’s OSSAM (adding Automation which is implied in every letter of OSSM)…. consumer electronics manufacturers haven’t really delivered the scalability components, but rather what seems to be an overprovisioned appliance or box. The cloud is not a box, nor a puppet show, but maybe more like a vending machine. Get served.

We might be engineers or developers but we’re often not a this-or-a-that we’re often both. And I think in DevOps culture this is the case. I think it’s also the case that a desktop hybrid microcloud can handle heavier video production workloads much better than a beefed up mac (request demo), due to parallel elastic provision at hyperscale supporting rendering workloads for example. And that’s just one example because rendering is just one video production workload. And when these guys get bored they play LAN parties which works really nicely with a desktop microcloud in your cube farm or wherever.

So think how software engineers play with supercomputers while video producers play 3-D shooters. It’s a competition, but for practical purposes the same infrastructure is used to prove the concept that collaboration is like competition on steroids… especially when you can use the same tools and share the same big data insights.

So at CES this year it really seemed as though cloud either meant wireless or SAN or NAS… but I think cloud storage is a nice low hanging fruit. Cloud persistence is the other benefit of microcloud. It’s a gateway to public utility persistence of files. So it takes the load off your tablets and keeps things locally accessible via ultra high speed bandwidth while it slowly persists remotely in heaven… eventually consistent and redundantly persistent… You can take it with you.

If that guy up there was a Patent Troll he would be taking 101 or 280 probably…actually yeah he might be taking the golden gate bridge maybe the other way. Maybe he’s trolling for honeys on USENET or IRC. Maybe he’s on his way home from stealing some ideas from the guru of all Marina yupster intelligence. Hopefully the good folks at GoGrid won’t let this troll near their datacenter if he heads back. I can’t tell you whether or not a famous Giant was using steroids, but I can tell you that this is not a San Francisco Giant on steroids carrying a club instead of a bat… and I can tell you that collaboration is like competition on steroids.

Automation, hated as it may be by some (service-providers usually), removes the human component. Self-service provides many access to something less constrained that only few experts could access before. If you try to pump your own gas in Oregon, you may be advised by an attendant. If you try to pump your own beer at the pub… they reserve the right to refuse service to anyone.

Even the most product-focused constraint theorists agree that at minimum viability, any product contains at least one service component. The theory of constraints doesn’t stop at identifying a constraint, but goes much further in exploiting the constraint often through systematic means. It’s systemic, however, when a service provider becomes the constraint, because efficiency can be greatly reduced by a single bottleneck.

The pattern suggests that service providers themselves are often the constraint and targeted by innovative (yet amateur) constraint theorists who aim to take the incumbent service provider’s position, but are not high-attenders to the theory of constraints because they fail to practice disciplined exploitation of the constraint. Wannabe constraint theorists fail to scratch the surface of the theory of constraints when they position themselves in such a way that they become the constraint. Value sustainability is questionable where a service provider becomes the constraint. For inventions (by value standards) the constraint must be exploited and the cycle must repeat. It proves the time-line itelf, in all its linear glory, to be the ultimate (unexploitable) constraint. This is why the most loved project managers keep developers and other contributors off the critical path and work within only the transparent actual deadline… the only real deadline… the only deadline which is capable of killing everyone on a given project. Some say whoever’s holding the clock’s holding the keys, but they better be holding my keys if it’s on my clock this new year, because I’m going to need another drink and another deadline before the fog clears and the cloud is recognized as more than just smoke and mirrors.

In application / software development and other technology project lifecycles: to become the constraint is to become eventually obsolete… consistently. This is just one example of why planned obsolescence beats the unplanned alternative.

Constraint Exploitation Spirals as a Delivery Product for Immediate Cross-industrial Consumption and Consortium

The theory of constraints becomes even more interesting when it’s applied to open-source vs. software-as-a-service or DevOps methodologies vs. Design practices (constraint invention) or from a more generally abstract perspective: Nature vs. Nurture (DevOps applied to fields outside of software engineering such as business development / Biz DevOps) … other bastardizations follow, integrations evolve and chapters merge. Thoughts converge and in iteration constraints expire to evolve design practices within an actual framework of iterative methodologies which function as a spiral of threads into various fields of expertise and operation scale. At the constraint, the spiral is most tightly woven, but as it reaches endpoints in the periphery of the market, iterations unwind and loosen the coupling for lower attendance. This is a process by which service constraints may be woven and unraveled to produce value between what-is and what-could be. Reducing friction between what-is and what-could-be is the secret weapon DevOps methodologists are now bragging about in between sword fights over language or configuration management tool manufactions. The delivery mechanism empowers the inventor or innovator to design, architect, implement, or otherwise become part of a constraint which sells itself to a market governed by global open standards.

In 2011 many were still wondering if the CLOUD really meant anything in terms of technology, dollars, and or cents. Looking back on 2011 all I can see is a whirl of nebulocity surrounding what-is with what-could-be. Here’s what I think might change significantly in the next 12 months or so:

The CLOUD is real… WHERE’S MINE?

Ok, so we’ve seen people make money off cloud… now I want one. Go build me my own thing that makes money too. Make it look like the King and maybe the King will be forced to buy it… I mean there can’t be 3 kings can there? So now that 2012 is almost here… people are realizing the cloud isn’t just a nebulous swirl of vapor-ware… now let’s start the ASP second chance foundation. Do I need a license for that? I think there will be a lot of opportunities to abstract licenses with SaaS deliveries. Some may exploit the gimmicks that should not have been codified into the licenses in the first place. What comes around goes around, but by now the only ISVs who are likely to be affected by it are the monolithicly most comprehensive solution providers who claim they invented everything. Invention by consolidation should be on the rise in 2012, by the way, I’m guessing.

ASP Second Life

Application service providers were right. Applications can often be served better warm, with human love. At minimum viability, a product contains at least one service component. Automation is great, but services contain humans and humans contain human error. Consumers love to cut out the middle-man, but once they’ve made all their man-in-the-middle attacks and all their paper dolls of sliced and diced middle-men they realize that they want service. So they go to http://asherbond.com/contact and ask for technical advice. Anyone who knows Second Life (or other virtual realities) knows that people like to design things and build things themselves. But if you’re going to build a cloud please ask yourself where the economies-of-scale exist. Now that the technology concepts have been proven in business practice many more customers are going to ask for cloud service, but what they’re really actually asking for is people (sometimes via a RESTful API).

The difference between application services and software-as-a-service is abstraction measured by a degree of multi-tenancy.

Compliance-and-Regulatory-Tunneling-and-Channeling-as-a-Service

They thought regulations and compliance “hurdles” created jobs… and they were right… in the short term… but what they might have missed is that it also creates jobs for service providers who can broker emerging technology as a service.

Business-Process-as-a-Service (#BPaaS)

What kinda cloud u talking bout? We got SaaS BPaaS and my personal favorite: GSaaS. GSaaS loves you brother. Now let me show you how to run your business. I expect to hear a lot of “what kinda PaaS” from developers and a lot of ooooo aaaah from business process practitioners… but the process consultants deserve a chance to really shine and this is it. I got my developer card revoked a couple times for saying “Cloud is SOA” but I got a new one from VeriSign and now I think developers are starting to be cool about it now that they realize that OASIS was right and that so was I since I said so too, neh. The first guy who raked my graphic depictions over the campfire did admit however, “yeah ok man.. i guess if you’re talking about REST.” So it turns out predictions in 2010 were accurate. I think service-component architecture and visual programming are going to play a role in RESTful integration as software components are service-oriented. I strongly expect scalability requirements and cloud-readiness motivators to stir the pot. Service-orientation is inevitable when technology is applied. Developers are empowered as decision makers and technical advisors, so maybe they would be interested in subscribing to business-process-as-a-service since they have more of a technical focus.

The most COMPREHENSIVE solution – brought to you by the Federated Association of Governing Consolidators

So what if you’re an investor and you buy and sell technology securities and you want some of that good old fashioned ROI. How can you make any money in this cloud biz now that the developers are taking over? Oh yeah there’s this little thing called the most COMPREHENSIVE solution. Big comprehensive, little solution. That’s right folks. The time is NOW. Buy everything. Your cloud portfolio is about to make it rain, but before you buy everything… you have to know how this stuff works and what it does. Haha just joking… now back to our regularly consolidated program… I think in 2012 we might continue to see enterprisey comprehensive solution providers trying to convince people that they are the box you can put your cloud into… or are they more of a comprehensive solution “cloud” that spans actual clouds with meaningful definitions which exist in actual physical datacenters? Who gives these large enterprisey comprehensive solution providers the authority to do this? The customer lets them get away with it because they sponsor industry events and they are often older companies who played a role in many of the technologies that end up as cloud. They equivocate between distribution models of cloud computing, for example… they might get behind the technology curve doing tons of non-emerging has-been-mature-for-a-decade-or-so SaaS business then pretend they are powering IaaS today on a public scale… when the emerging technologies are PaaS based.

DevOps as more of a cultural paradigm shift and movement and less of a title

People are going to start either killing each other based on their choice of configuration management / automation framework or they are going to start getting along more and not putting DevOps in their title unless it has Engineer at the end of it and Lead in the the front of it. Designers are going to be constrained by tighter iterations and Ops are going to punch developers just because they haven’t been punched before and everyone goes through it.

Developer-as-a-customer

In the old days, developers could be divided and conquered by business managers much more easily. The days of developers having a great idea that no one understands are not over… but “I don’t understand how this stuff works” is no longer an excuse now that we have so many services available. If you don’t know how something works… just ask… only now… you don’t even have to ask how to do it, you can ask for service. If you don’t know how something works, that something might be new and valuable. Dustin said it already, but I think public offerers are going to focus more on influencing the decisions of software developers. Software developers represent change in the direction of requirements and demands… not just whatever seems wanted right now… I think developers often try to guess (like Steve Jobs R.I.P.) what people need since they’re probably going to want that eventually. I could probably guess that a pregnant mom is going to be in the market for diapers sooner or later. Hopefully sooner rather than later. Developers are in the early stages from cradle to grave. They iterate through software development and application life cycles and deliver features based on requirements. Those features become part of a common framework that can be offered more publicly. It’s not new, but software vendors love to put developers on their platforms. What’s new is that developers are not-so-divided and not-so-conquered… so they probably demand a higher degree of ubiquity in their distribution channels… so they probably demand a higher degree of interoperability in their language frameworks.

Applications are most portable when the target distribution platform is based on open-standards.

Public Platform-as-a-Service (PaaS) Top Doggery

Not everyone can be King of the Hill, but I think there’s room for a whole circle of winners in the market segment of public PaaS. We have seen 3 generations of public platform service offerings to developers:

Totally Rigidly Arcane PaaS

The first platform services with public offerings forced the developer to conform to a proprietary framework. The back end was a confidential operation delivered as a multi-tenant service to subscribers who learned how to conform to the proprietary framework. The framework may have been based on python or java, but constrained the developer to the platform of implementation rather than the standards of the enabling technologies within.

Still-exploiting-the-constraint PaaS

This type of platform is built secretly and operates as a proprietary service, but relies on open-source components to deliver services which are mostly compliant with open-standards. A true language is always an open-standard.

Open PaaS – as it should be

Third generation platform services are completely portable. This type of middle-ware essentially replaces the role of the “operating system” as a software component with “systems-in-operation” instantiated as objects by a framework of classes delivered as a platform of services for developers to build things on top of. The distribution model allows for services to be delivered with scalability, flexibility, interoperability, high availability and the distribution model also allows for platform portability and application interoperability by default. The evolution of service-component architecture (SCA) and visual programming may also influence the adoption of visual programming in the cloud as practical users are abstracted by service and frictionless design becomes the practice.

Next Generation PaaS+

I think of PaaS+ as a value-added platform-as-a-service which may include business processes as a service or may include additional DevOps tooling or methodologies-as-a-service (MaaS?) whatever… The framework (tool) teaches you the process. In a toolcloud you might experience something like a toolbox… for example when you’re using Gmail, you realize that Gmail is a Google approach to email… it’s not just an “email program” … so you get some agility along with the nebulocity of the cloudy SaaSfulness. So I think that the next generation PaaS+ will need to put their pluses on by adding some kind of business or other practical high level value. Some of this high level value can be delivered in the form of integration. Cloudbees has moved forward with their initiative to add continuous integration via Jenkins/Hudson integrated service components in their PaaS offering. I think DevOps toolclouds will emerge via the PaaS delivery model and that like Cloudbees other cloud service providers who have a PaaS offering may choose to offer a chocolate or strawberry new flavor of PaaS for Dev and possibly a vanilla PaaS for their long term support in production interoperability and highly available portability PaaSes. I guess Leiloo Dallas could call that one a multi-PaaS just in time to kiss Korbin and save the world before New Years.

Predictive Monitoring and SLAs

Predictive monitoring tools will leverage Hadoop and other big data / analytics. The abstraction of data itself may become an abstract business-process-as-a-service and drive innovation in system performance as SLA’s are enforced and predictive deep monitoring tools allow autonomous and dynamic autoscaling of instances in resource pools.

Resource Pool Expansion and Utility Computing Commodotitization

I think the price of public cloud will start to look like a true utility and come down quite a bit. Companies like Amazon Web Services probably would lower their prices is the demand wasn’t way too high. When more IaaS vendors such as Rackspace, Opsource, Datapipe, et al.. enter the space (they’re already here) and start to compete for customers, the price of raw x86 compatible IaaS should come down quite a bit and make people re-think their hybrid strategies. For now, many organizations may benefit from a flexible hybrid cloud strategy that (for example) may leverage their existing infrastructure to orchestrate public cloud services.

Security implications of Cloud Computing

Cloud computing lowers the barriers to entry by people who ordinarily could not access high performance clusters of nodes to do complex brute-force math research on your “encrypted” password… or just fire up an array of nodes and aim it at the ssh port. Nothing they couldn’t do in the old days of dark matter / botnet clouds. What IP address did that come from? A leased one in a classy datacenter. I think public cloud providers are going to become very security-savvy (actually they really are top notch in most cases). It will be interesting to see how they empower themselves from the big data + hypervisor perspective.

Rinse that CLOUD out ‘cha mouth boy!

At some point… analysts are saying that there is a “hype cycle” in which cloud word sentiment shall become stale. The word cloud will either become ultra-ubiquitous like industry insiders are saying… or it may become a bit blase.. numb from the excessive nebulocity of smoke and mirrors becoming clouds too. I think if we can refrain from partying too hard it might help. Happy new years eve. Be responsible and make backups.

Is Context King?

A global positioning system in a networked mobile device can provide a narrower data scope (based on coordinate tracking) to an event handler or software controller which queries a cache of temporal data or even a larger set of persistent data. When focused views of user defined human-relational data sets intersect dynamic geospacial coordinates, systems can more efficiently learn how to provide more relevant information about changing social contexts. Moreover, this process can be done with less manual input by users. As a mobile user moves through geographic space, his or her social context may change based on the absence or presence of other people. As distributed systems and social networks become more aware of frequently changing, subtle, geographic social contexts… it becomes increasingly possible (assuming the information is shared with users) for users to find places and people based on their interests. From a social perspective, it’s really quite empowering to have this much abstraction between a venue and a place. There are a lot of “cloud” companies around these days, but in order to really specialize in abstraction one must understand what becomes the focus instead of that which we abstract. The focus is where the actual power (value) comes from. In this case the power is in the ability to become less reliant upon physical, geographic constraints and more focused on social interests (whether more or less sophisticated) and more focused on relateable interactions.

For example, if Barack Obama is in town, perhaps I’d like to invite him as a guest to an upcoming local or regional event. If not, perhaps I’d like the system to invite the next person who might be interested (in this example… Tom Anderson, my first Myspace friend). The system can know (often based on what’s probably voluntary user input) that Rick Perry isn’t as interested in this particular event (perry-wink)… even though both Rick Perry and Barack Obama may exist in the same or similar regional space at the same time. But fortunately I still have my friend Tom. The converging technologies (big/aggregate data abstraction, mobile computing, geospacial contextualization, and social contextualization) are not so new when standing alone, but when integrated support a trend of social sophistication which is more agnostic of physical infrastructure and places where it exists. If I had to summarize this sophistication in one word it would probably be “freedom.”

Skeptics might argue that they don’t like the idea of software or systems knowing more about them. Skeptics might also argue that they don’t like knowing << unpleasant fact(s) >>. Often, an unpleasantness (fear) is associated with aggregation of knowledge within a technological or otherwise sophisticated framework. You could call it a “fear of singularity” or “fear of robots taking children somewhere else” … but this year and probably next year I’m likely to be more afraid of ignorance than artificial, collective, or social intelligence. Knowledge is power, computers are tools, and the more they know about us the more we can know about ourselves. Privacy is important and made possible if encryption and private ownership of data is made possible. You are free to navigate your social context. Is your data? Is “your” data your data?

Peep the Context.

On-demand self-service

A consumer can unilaterally provision computing capabilities, such as
server time and network storage, as needed automatically without requiring human
interaction with each service’s provider.

Broad network access

Capabilities are available over the network and accessed through standard
mechanisms that promote use by heterogeneous thin or thick client platforms (e.g.,
mobile phones, laptops, and PDAs).

Resource pooling

The provider’s computing resources are pooled to serve multiple consumers
using a multi-tenant model, with different physical and virtual resources dynamically
assigned and reassigned according to consumer demand. There is a sense of location
independence in that the customer generally has no control or knowledge over the exact
location of the provided resources but may be able to specify location at a higher level of
abstraction (e.g., country, state, or datacenter). Examples of resources include storage,
processing, memory, network bandwidth, and virtual machines.

Rapid elasticity

Capabilities can be rapidly and elastically provisioned, in some cases
automatically, to quickly scale out, and rapidly released to quickly scale in. To the
consumer, the capabilities available for provisioning often appear to be unlimited and can
be purchased in any quantity at any time.

Measured Service

Cloud systems automatically control and optimize resource use by leveraging
a metering capability at some level of abstraction appropriate to the type of service (e.g.,
storage, processing, bandwidth, and active user accounts). Resource usage can be
monitored, controlled, and reported, providing transparency for both the provider and
consumer of the utilized service.

After attending a few Cloud Camp events I had the privilege of discussing what cloud computing is with Dave Nielsen. Here is the OSSM (pronounced awesome) CloudCamp definition of cloud computing Dave Nielsen has been presenting:

On-demand: the server is already setup and ready to be deployed
Self-service: the customer chooses what they want, when they want it
Scalable: the customer can choose how much they want and ramp up if necessary
Measureable: there’s metering/reporting so you know you are getting what you pay for

While I really couldn’t dispute the awesomeness of Dave’s definition, I challenged it. I just felt the need to add automation to the definition. Here is the OSSAM definition I came up with:

On-demand

Architecture is implemented by an operating framework that allows for rapid elasticity. This framework determines which hardware and software resources are required to meet a range of service-level agreements and subscriber (i.e. customer) expectations.

Scalable

Scalability is achieved on the back end via tight and loose coupling of hardware resources, orchestrated to meet the changing demands of different use cases. A grid may provide the computational and storage resources or a network of edge caching servers may provide content distribution. Many public cloud providers offer both. On the front end, virtual and paravirtual machinery provides subscriber-facing service nodes powered by an elastic hardware and network infrastructure layer (resource pool) of computational nodes and storage area networks on the back end. Vertical scaling is limited to the capacity of one piece of today’s best hardware, but cloud scalability means that arrays of nodes can be offered a service layer or unit… which provides horizontal scalability, rapidly on demand.

Self-service

A multi-tenant framework must exist to provide at least two tiers of service layers. The underlying infrastructure tier represents technical operations and the top tier(s) represent one or more abstracted service layer(s) … oriented to providing services specific to an applied scope of operations. (For example, a business use case for a particular department or organizational unit). Self-service also means that you have the ability to manage your own service layer(s) if that is how you, the subscriber, decide to provision your resources. For example, if someone is a systems administrator he or she may decide to provision a computer in the cloud with or without a managed operating system or with or without the management of a software library layer. Commodity virtual-machinery which is unmanaged is an example of this, but certainly a fully managed virtual machine could fall under the category of ‘self-service’ if the subscriber tells the API to provide them a managed virtual machine. This leads to my bastardization of the Cloudcamp definition…

Automated

There must be some degree of automation for cloud computing to be a true “vending machine” and more than just a puppet show. When an order is placed, human resources, engine-squirrels, or monkeys must not be employed to carry out the provisioning of services… no matter how rapidly they may be able to provide services. The system must automatically, via an API be able to mechanically provide services within the range of some kind of service-level agreement. While it’s arguable that this is part of “on-demand” services, I think it’s worth making a distinction. The importance is that on a large scale, on-demand services can’t exist without automation. Fail-over at the hardware level, for example, may not be required for cloud computing to be defined… but fail-over is a crucial piece of the puzzle if storage and compute nodes within a cluster are to provide reliable and sustainable support for complex layers of virtualization.

Measureable

Metering and reporting is important not only for billing in public cloud service implementations, but also in private clouds which service enterprise departments. Measurement provides a quantitative analysis of resource utilization and allows for more efficient use of computational resources. On the “front-end” metering tells subscribers how much resources they are consuming and on the “back-end” measurement should also tell infrastructure operators when to add hardware resources to the computational / storage grid. With proper fail-over, automation and orchestration these resources should be highly available and a monitoring system should measure that availability.

The separation of static content from dynamic content is a concept that will lead to higher efficiency and higher availability in any Cloud environment. Backups from master to slave databases may seem expensive across availability zones, but perhaps, after today, they are less expensive than we once thought.

Now let’s think about Content Distribution Networks. Static content can be cached at the edge which provides the most availability to your end users. When people think of CDN availability, they might assume “closest geographical region to the end user”… but what if your CDN was smart enough to weigh latency and system load as metrics in the load balancing determination algorithms? Do we have that? Yeah. Skeptics blame AWS / EC2 for today’s hardships, but perhaps some should be thanking them for edge-caching static content worldwide. It’s a saving grace for those who have their eggs scattered amongst 18 geographic regions.

For static content, content distribution networks often have multi-region high availability built in out-of-the-box. It’s a lot easier when dealing with static content, but with some systems architecture and database management expertise, the same caching principles can also be applied to maximize reliable delivery of both static and dynamic content.

If an application provider or platform service provisioner can separate static content from persistent data and also separate important data from not-so-important temporary / session data and deliver these types of data and content with discardable instances… fail-over can be achieved and even automated by replicating data across providers (or at least across cloud regions / availability zones). Once static content, persistent data, and temporal data have been sorted out… a redundant, meshed / multi-homed front-end server-array tier can determine (based on monitoring and availability metrics) which cloud / data-center / availability zone to distribute static and dynamic content from.

I think this type of architecture can be justified not only for fail-over reasons, but perhaps it can also be a way to achieve more rapidly elastic, impressive server performance.

When N. Virginia gets hit hard, it may be quite a hardship, but it shouldn’t be too hard to fail over to your other region’s slave database. If Soichiro Honda is going to tell me that success is 99% failure, then in the case of distributed, edge cached, redundant web systems architecture… perhaps success is 99% failover.

But don’t just go throwing shuriken at network-event coordinators unless your star has more than just these two points. I think a nice third point to sharpen and cut to is the reliability of monitoring systems. It’s good to be monitoring your auto-scaling processes if you’re in a situation where you scale on demand… and you also want to monitor who is demanding the computing resources. Ideally, you’re getting alarmed before your end users are. Reflexive firewalls are a good way to go, but just having good reflexes is part of wearing the agile cat’s hat in general. If you have a fast way to report trouble to the authorities charged with ownership of a compromised node attacking your system, you’re part of the solution and get a gold star.

Conversely, unnecessary reflexive post-mortem backups en-mass may have been a somewhat panicked response to the network event and a contribution to the length of this outage.

Amazon Web Services has done an excellent job (as always) of not only describing what happened and when service is expected to be restored, but what you can do to maximize availability if your service has been adversely affected by the outage. You can access their status updates via RSS feeds directly from the AWS Service Health Dashboard at status.aws.amazon.com.

Here’s a copy of what AWS is saying about EC2 services in the N. Virginia region [ RSS ]:

1:41 AM PDT We are currently investigating latency and error rates with EBS volumes and connectivity issues reaching EC2 instances in the US-EAST-1 region.
2:18 AM PDT We can confirm connectivity errors impacting EC2 instances and increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region. Increased error rates are affecting EBS CreateVolume API calls. We continue to work towards resolution.
2:49 AM PDT We are continuing to see connectivity errors impacting EC2 instances, increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region, and increased error rates affecting EBS CreateVolume API calls. We are also experiencing delayed launches for EBS backed EC2 instances in affected availability zones in the US-EAST-1 region. We continue to work towards resolution.
3:20 AM PDT Delayed EC2 instance launches and EBS API error rates are recovering. We’re continuing to work towards full resolution.
4:09 AM PDT EBS volume latency and API errors have recovered in one of the two impacted Availability Zones in US-EAST-1. We are continuing to work to resolve the issues in the second impacted Availability Zone. The errors, which started at 12:55AM PDT, began recovering at 2:55am PDT
5:02 AM PDT Latency has recovered for a portion of the impacted EBS volumes. We are continuing to work to resolve the remaining issues with EBS volume latency and error rates in a single Availability Zone.
6:09 AM PDT EBS API errors and volume latencies in the affected availability zone remain. We are continuing to work towards resolution.
6:59 AM PDT There has been a moderate increase in error rates for CreateVolume. This may impact the launch of new EBS-backed EC2 instances in multiple availability zones in the US-EAST-1 region. Launches of instance store AMIs are currently unaffected. We are continuing to work on resolving this issue.
7:40 AM PDT In addition to the EBS volume latencies, EBS-backed instances in the US-EAST-1 region are failing at a high rate. This is due to a high error rate for creating new volumes in this region.
8:54 AM PDT We’d like to provide additional color on what were working on right now (please note that we always know more and understand issues better after we fully recover and dive deep into the post mortem). A networking event early this morning triggered a large amount of re-mirroring of EBS volumes in US-EAST-1. This re-mirroring created a shortage of capacity in one of the US-EAST-1 Availability Zones, which impacted new EBS volume creation as well as the pace with which we could re-mirror and recover affected EBS volumes. Additionally, one of our internal control planes for EBS has become inundated such that it’s difficult to create new EBS volumes and EBS backed instances. We are working as quickly as possible to add capacity to that one Availability Zone to speed up the re-mirroring, and working to restore the control plane issue. We’re starting to see progress on these efforts, but are not there yet. We will continue to provide updates when we have them.
10:26 AM PDT We have made significant progress in stabilizing the affected EBS control plane service. EC2 API calls that do not involve EBS resources in the affected Availability Zone are now seeing significantly reduced failures and latency and are continuing to recover. We have also brought additional capacity online in the affected Availability Zone and stuck EBS volumes (those that were being remirrored) are beginning to recover. We cannot yet estimate when these volumes will be completely recovered, but we will provide an estimate as soon as we have sufficient data to estimate the recovery. We have all available resources working to restore full service functionality as soon as possible. We will continue to provide updates when we have them.
11:09 AM PDT A number of people have asked us for an ETA on when we’ll be fully recovered. We deeply understand why this is important and promise to share this information as soon as we have an estimate that we believe is close to accurate. Our high-level ballpark right now is that the ETA is a few hours. We can assure you that all-hands are on deck to recover as quickly as possible. We will update the community as we have more information.
12:30 PM PDT We have observed successful new launches of EBS backed instances for the past 15 minutes in all but one of the availability zones in the US-EAST-1 Region. The team is continuing to work to recover the unavailable EBS volumes as quickly as possible.
1:48 PM PDT A single Availability Zone in the US-EAST-1 Region continues to experience problems launching EBS backed instances or creating volumes. All other Availability Zones are operating normally. Customers with snapshots of their affected volumes can re-launch their volumes and instances in another zone. We recommend customers do not target a specific Availability Zone when launching instances. We have updated our service to avoid placing any instances in the impaired zone for untargeted requests.
6:18 PM PDT Earlier today we shared our high level ETA for a full recovery. At this point, all Availability Zones except one have been functioning normally for the past 5 hours. We have stabilized the remaining Availability Zone, but recovery is taking longer than we originally expected. We have been working hard to add the capacity that will enable us to safely re-mirror the stuck volumes. We expect to incrementally recover stuck volumes over the coming hours, but believe it will likely be several more hours until a significant number of volumes fully recover and customers are able to create new EBS-backed instances in the affected Availability Zone. We will be providing more information here as soon as we have it.

Here are a couple of things that customers can do in the short term to work around these problems. Customers having problems contacting EC2 instances or with instances stuck shutting down/stopping can launch a replacement instance without targeting a specific Availability Zone. If you have EBS volumes stuck detaching/attaching and have taken snapshots, you can create new volumes from snapshots in one of the other Availability Zones. Customers with instances and/or volumes that appear to be unavailable should not try to recover them by rebooting, stopping, or detaching, as these actions will not currently work on resources in the affected zone.

10:58 PM PDT Just a short note to let you know that the team continues to be all-hands on deck trying to add capacity to the affected Availability Zone to re-mirror stuck volumes. It’s taking us longer than we anticipated to add capacity to this fleet. When we have an updated ETA or meaningful new update, we will make sure to post it here. But, we can assure you that the team is working this hard and will do so as long as it takes to get this resolved.

Notice the ENTIRE CLOUD has certainly not collapsed. They are providing you a way to spin up instances in many availability zones that are available as usual. These are highly available availability zones which are not affected by this outage and may serve as failover with proper implementation of redundant server architecture.

Here’s a copy of what Amazon Web Services is saying about RDS services in the N. Virginia Region [ RSS ]:

1:48 AM PDT We are currently investigating connectivity and latency issues with RDS database instances in the US-EAST-1 region.
2:16 AM PDT We can confirm connectivity issues impacting RDS database instances across multiple availability zones in the US-EAST-1 region.
3:05 AM PDT We are continuing to see connectivity issues impacting some RDS database instances in multiple availability zones in the US-EAST-1 region. Some Multi AZ failovers are taking longer than expected. We continue to work towards resolution.
4:03 AM PDT We are making progress on failovers for Multi AZ instances and restore access to them. This event is also impacting RDS instance creation times in a single Availability Zone. We continue to work towards the resolution.
5:06 AM PDT IO latency issues have recovered in one of the two impacted Availability Zones in US-EAST-1. We continue to make progress on restoring access and resolving IO latency issues for remaining affected RDS database instances.
6:29 AM PDT We continue to work on restoring access to the affected Multi AZ instances and resolving the IO latency issues impacting RDS instances in the single availability zone.
8:12 AM PDT Despite the continued effort from the team to resolve the issue we have not made any meaningful progress for the affected database instances since the last update. Create and Restore requests for RDS database instances are not succeeding in US-EAST-1 region.
10:35 AM PDT We are making progress on restoring access and IO latencies for affected RDS instances. We recommend that you do not attempt to recover using Reboot or Restore database instance APIs or try to create a new user snapshot for your RDS instance – currently those requests are not being processed.
2:35 PM PDT We have restored access to the majority of RDS Multi AZ instances and continue to work on the remaining affected instances. A single Availability Zone in the US-EAST-1 region continues to experience problems for launching new RDS database instances. All other Availability Zones are operating normally. Customers with snapshots/backups of their instances in the affected Availability zone can restore them into another zone. We recommend that customers do not target a specific Availability Zone when creating or restoring new RDS database instances. We have updated our service to avoid placing any RDS instances in the impaired zone for untargeted requests.

11:42 PM PDT In line with the most recent Amazon EC2 update, we wanted to let you know that the team continues to be all-hands on deck working on the remaining database instances in the single affected Availability Zone. It’s taking us longer than we anticipated. When we have an updated ETA or meaningful new update, we will make sure to post it here. But, we can assure you that the team is working this hard and will do so as long as it takes to get this resolved.

These updates are not direct from Amazon, but merely a copy, so please subscribe to the Amazon Service Health Dashboard for more freshly updated information regarding their service (which I still insist is high quality).

- Asher Bond
It’s a long way down if your head is in the CLOUD.

I’ve had difficulty quarantining the feature-itis, because I can’t stop dreaming about integrating Twitter and Facebook differently.

What I envisioned in these dreams (which I hope you won’t call nightmares) is something I’m calling an ‘integration aspect’ which would display the contents a Twitter account, Facebook wall, etc. This would be different from the way it is now: limited to a single Facebook wall or a single Twitter account only accessible as write-only when posting to the world. I think it would be useful to read and write to ‘integration aspects’ which are Twitter accounts, Facebook Walls and Facebook Page Walls, and whatever API hackers deem hackable in their integration adventures. ‘Integration aspects’ would become part of the ‘all aspects’ aggregation and who knows, maybe even serve as a way to drag and drop share invitations from Diasporans to Twitter and Facebook, et. al.

You might be asking yourself, “Why do we need to support multiple Twitter accounts?” Perhaps we don’t NEED it, but many people have at least two accounts, especially for business purposes. In some ways, these multiple accounts are their attempt at ‘aspects’ in the world of Twitter.

You might be asking yourself “Why do we need to support more than one Facebook wall?” Facebook pages have their own walls which are manageable under one account.

The presence of SaaS dashboards which already integrate these social network features… and the fact that some of them charge $5 to $15 per month per user… perhaps indicate demand.

You might be asking yourself “How on earth and in Diaspora would we execute these features without causing API drama?” That’s a good question. We would most likely need to heavily cache the results from querying these social networks at large. It’s doable and others have done this with their integration platforms.

The value proposition is that these features would create a multi-threaded integration environment which is consistent with the aspect-driven, contextually-aware user experience we have already instigated.

- Asher Bond
asherbond@joindiaspora.com
https://convore.com/diaspora/ui-ux-uask/

A step-wise approach to changing the world of social networking forever and ever:

1. Go to https://github.com/diaspora/diaspora
2. Click “Fork this.”
3. Sign up for access to a pod and provide alpha feedback
4. Push your secret sauce to master if you’re dev savvy.
5. Merge updates from the main project and others if you’re dev savvy.
6. If you’re design savvy be providing lots of feedback via getsatisfaction and via Github issues”
7. If you’re a social butterfly invite 25 high attenders to your Diaspora aspects
8. If you’re a deployment butterfly (lol you’re a moth) then make sure the chef recipes are in good alpha health?
9. DiPaaSpora shall be upon you.. heroku tonight.. tomorrow THE WORLD of Open PaaS.
10. If you’re a maker… make some cheap pods under your desk and see if you can move your contacts… soon you will be able to move (import+export) all of your data
11. Join the mailing lists. There is one for dev, one for services-integration, and of course the standard user group for everyone.
12. Go to the diaspora meetups… when people ask you when Diaspora is going to launch or take off..

There are more characteristics of cloud computing than just automation and autonomy or other methods of self-provisioning. Cloud computing exposes APIs that allow subscribers to access computational resources on demand which serve as an abstraction layer between aggregated (and possibly also distributed) hardware and virtual (or paravirtual) machinery. A proudly provisioned Cloud usually boasts some kind of synergistic automation, monitoring, distribution, and compute aggregation at every service layer. The customer experience should be a burst-friendly, high availability, easy to use, “just-works” success story. Nobody likes when the vending machine takes money then gets jammed or you find out that the people inside your TV and radio are really there looking back at you and listening to all your secrets. So again, the Cloud isn’t a puppet show, but more of a vending machine. Get Served.

Finally a social media dashboard that integrates your LinkedIn, Facebook, Twitter, Myspace, Tumblr, and other social networks et. al. into one mobile-friendly platform that let’s you sign on once and manage many social media campaigns and social media teams. Multi-tenant and multi-platform? Yeah.

Social Media Analytics in Hootsuite

HootSuite Social Analytics from HootSuite on Vimeo.

(Music: Salteens / Grey Eyes – Everything They Know About Us)

Elastic Provisioner is proud to announce that we are now a HootSuite Pro Solution Partner.

HootSuite is a social media management system for businesses and organizations to collaboratively execute campaigns across multiple social networks from one secure, web-based dashboard. Key social network integrations include Facebook, Twitter, and LinkedIn, plus a suite of social content apps for YouTube, Flickr, Tumblr and more.

Earlier this year HootSuite hit 3 million users with over 700 million messages sent through over 6 million unique social profiles. Along with HootSuite’s web platform, 20% of users access the dashboard through their mobiles including iPhone, Android, Blackberry and iPad. HootSuite also offers localized versions of their dashboard in six languages – English, French, Italian, Japanese, Spanish and Portuguese.
There are many benefits to HootSuite Pro

Engage: Optimize your audience engagement by creating search streams, scheduling messages and monitoring all of your social network profiles from one customizable web and mobile dashboard.

Collaborate: Invite clients and colleagues to participate in your social media management. Assign messages for follow up and share streams, helping you increase efficiency.

Analyze: Measure your efforts using over 40 social analytics modules to build and share custom reports. Or select from one of our pre-made templates for quick and easy reporting.

Secure: Share access with team members without compromising security. The team permission levels and advanced sharing options ensure you remain in control of your valuable social profiles and accounts.

Sign up for a 30-day free trial of HootSuite Pro now: http://ow.ly/3C8qv