PaySwarm Beta Launched

This wasn’t a typical Beta launch… it wasn’t what we intended to do when we started.

We have been focused on creating an open, universal payment standard for the Web for the past 24 months. In late February, we launched an alpha release of the PaySwarm reference implementation with a plan to release a commercial product by the end of this year based on the the reference implementation software. The guts of the system was built on a previous product that we had launched in the music industry for doing legal peer-to-peer music sales. The software was very fast, had achieved stability over the past 5+ years, and was familiar to our development team.

Ghosts of the Past

Our reference implementation software architecture was a mixture of C++ and Python with the database infrastructure using MySQL. The system was designed to scale, was well documented, and was the workhorse for a variety of the products that we had launched as a company over the past several years. Right after the PaySwarm Alpha launch in February, we walked away from our entire software architecture. We burned it to the ground. All of it.

Normally, this is one of those fundamental mistakes that many seasoned veterans in the start-up space will tell you to never make. Never re-write a system that is already more-or-less working for you. Especially don’t do it when you need to ship a product within the next six months. Don’t do it when you’re under-staffed. Just don’t do it. We did it anyway, and here’s why.

Our previous product needed to ship on Windows, Mac OS X, and Linux. It needed to run as a browser plugin. So, C++ was the best choice at the time. Our software would fire up a tiny Web server and allow control over our product through web pages served off of the same machine that was running the browser. When we switched to doing the universal payment standard, we re-used a large amount of the code base to speed development along. After a few months, a few things started to become apparent.

Cracks in the Hull

The first was that it was taking us an inordinate amount of time to integrate new features. While this may seem obvious in hind-sight, having to iterate features that were experimental in C++ had a very large per-feature cost. Creating a universal payment standard is not simple and we had to create many failed experiments before we whittled the solution to the problem down to something that was workable. C++ was restricting our agility, even though we had designed the system to be fairly agile. For example, we had modified the programming environment to use generic data objects and reference counting as much as possible. This is analogous to using generic objects in Python and JavaScript, where you can extend the attributes of the object at run-time. The whole build-run-test cycle eats away at your productivity, no matter how good your tools are. Building and re-building software takes a large chunk of time out of your day. The system had so many layers of functionality that we often spent more time doing maintenance than integrating new features.

The other issue that we were getting increasingly concerned about was scalability of the database cluster. Sure, MySQL is tried and true and we had a fairly decent scaling story. We were certain that we wouldn’t have to scale to billions of transactions a year at first, but it was transitioning to a design that could support it in the future was something to be concerned about. We decided to put the upgrade to the database architecture off to a point where we could see success, choosing to not over-engineer the first product in the hopes of saving some time to market. We had also benchmarked the C++ web service architecture we had and were faster than anything currently on the market at the time, so the speed of the web servers ability to handle large traffic loads was not a concern for us.

So, the biggest issues were agility and future database scalability, but we felt that we had everything else pretty well sorted. It was during 2011 that our CTO, Dave Longley, had taken a casual look at Node.js and MongoDB. He had been fiddling with the two systems as a side project, to see if we could accomplish lock-less atomic financial transactions in MongoDB. It had taken us close to five years to write the C++ code and get the code base stabilized to the point that we felt comfortable with it. We estimated that it might take 6 months to do a complete re-write in a higher-level language, which was clearly longer than we needed it to take in order to ship a product. We had decided to attempt a port to Node.js and MongoDB, but after we had done the first commercial release. That was the plan.

Sink it and Start Again

In parallel, we have been developing a new standard called JSON-LD. The PaySwarm specifications need JSON-LD in order to pass messages between various Web Services on the Internet. One of the hardest problems that we’ve had to deal with has been normalizing JSON-LD data so that we can perform a digital signature on that data. This has consumed months of engineering effort, required many very difficult hours of highly technical problem solving. Almost all of this mental effort and engineering time has been put in by our CTO. With burn-out caused by the JSON-LD work on the horizon and in a fit of frustration, Dave decided to switch projects for a little while to try out some ideas that had been rattling around in his head related to Node.js and MongoDB. A few weeks of work demonstrated that the port was going to be easier than we had thought. Dave didn’t mention this to the team.

A few weeks later, during a fairly standard demo of the software, Dave worked his way through our entire testing procedure. All of the screens were operating as they always had – our C++ software was as stable as it had been in the past. When the demo was finished, Dave turned to us and stated: “That’s our entire system, implemented in Node.js and MongoDB – all of it works.” Our jaws hit the floor – we had no idea that the entire system had been swapped out and replaced.

The best part about it was that it worked. It took us two more weeks to fine tune a few things, but development was already several factors of magnitude faster than it had been with the old system. We now had a database back-end that was going to scale much more easily than the custom MySQL system that we had put together. Developing in Node.js is a breath of fresh air, and this is coming from a group of developers that frequently use Python.

PaySwarm Beta 1

The PaySwarm Beta launch that you see today is a basic PaySwarm reference implementation, running on Node.js and MongoDB. While you can’t tell from the website, the entire infrastructure is new and will allow us to add features, fix bugs, and move the PaySwarm universal payment standard forward at a much faster pace than we’ve been able to in the past. We have also released a preliminary, open source, PaySwarm client library called payswarm.js, that can be used to interact with a PaySwarm Authority. The library is currently incomplete, but still allows you to do all of the basic things that are required to list assets (such as e-books, music, video, and other digital content) for sale and perform purchases on the open, standards-based PaySwarm financial network.

We’re currently focused on load-testing the entire system and putting the finishing touches on the rest of the commercial site. The next step is a commercial launch of the PaySwarm system. While we don’t have a firm date on the soft launch, we think it will happen at some point in the next six months.

Trackbacks/Pingbacks