Understanding the Market for Software Platforms

Like many software architects, I’ve built quite a bit of framework code in support of applications because the design patterns I wanted to use were not present in the core language.  As a software engineer who cares for the whole life-cycle of the products I build, I’ve always been looking for the best way to get declarative programmers more involved in development and customization of applications.  In 1996 I had finished building a visual programming language called AVS/Express with a sophisticated data binding system.  I realized AVS would never market it as a horizontal software tool and yet I was intrigued by the power these designs might have in the broader marketplace.  Like others, I believed Java would be the next big thing in software engineering and luckily found a great company looking to innovate in Java web platforms, ATG.  We designed a page template language, an IOC framework called Nucleus and a sophisticated ORM solution called “data anywhere”, all of which were keys to ATG’s success as both a platform and a customizable e-commerce solution.  Despite my advice to create horizontal offerings with these APIs, these remain high cost, five figure “enterprise” products and lost out to free offerings Spring, Struts, Hibernate when those were developed.

Today, ATG makes money regardless but their customers must spend a lot of money on headhunters given how many unsolicited phone calls and emails I get looking for trained ATG developers.  I bet their current e-commerce business would be even better if more developers were trained on their system.  I also suspect they are feeling saddled by a large platform code base to maintain that hinders them as much as helps now.  The cheaper stuff is evolving faster because more developers are using them.

I joined Adobe in 2005 because I believed they could push these design patterns into robust platform software that would get broad enough adoption to be successful.  With Flash/Flex they had a competitive UI but still were largely ignored by mainstream business engineering companies.  They needed server connectivity, round-trip UI to database tooling and improved standards compliance to really advance software engineering efficiency in a significant way among the corporate developers.  At the time I had come to believe that big companies were the best way to build software platforms because they could make large quantity, lower cost products successful and had the resources to plan for long-term investments.  It seemed like a perfect fit.

During the first Adobe internal developer’s conference I attended, the theme was “Platforms” and I was encouraged early on.  Sadly, I learned many lessons of big company politics that led me to learn their limitations when it comes to innovation.  In Adobe’s case, they are all about platforms on the client but when you get to the server, it becomes a political mine field.   When I was hired, I was told Flex would be much cheaper than its low five figure price at that time.  Shortly after I joined with the merger just starting things changed.  My product would have a free version but would cost an even higher five figured price for an unlimited one CPU clusterable license.  I never liked that pricing strategy but recently it just got worse.  They dropped the free version and raised the price on the unlimited version by another 20%.   For a set of tools so widely applicable (forms, persistence, etc.) and evolving elsewhere simultaneously that’s price will ensure other technology evolves more quickly to fill this need and Adobe loses their last/best monetization vehicle for Flash.

While I still believe I was right about big companies being the best places to evolve software platforms, I now see their limitations more clearly.   A big company has a complex political landscape and the deeply hierarchical management structure makes it more difficult to make good engineering decisions unless the leaders understand the vision.  Platforms combine standards with meticulous design and must include an efficient process for rapid evolution to ensure the success of its solutions.   Big companies will always be tempted to use their leverage and momentum to steer technology projects away from the path of efficiency towards a tactical advantage.  This type of decision making is the path to brittle and slowly evolving tools infrastructure.  To me software engineering is still engineering first and foremost.  We build mission critical components just like other engineers which become important public investments so corporate misdirection and bungling of technology advancements which harm efficiency particularly pains me.

I’ve spent the last 8 months part time researching the state of the industry looking for the next big language – the one like Java was in 1996 – that would help us make even better, more solid and maintainable software designs going forward.  I want maximum portability from mobile to desktop to cloud.  I want to leverage all of the language idioms we’ve all learned, leverage all of the library code we’ve built but improve the design integrity, flexibility and robustness of the designs.  I’ve looked at Ruby, Scala, JavaFX – the three major contenders and found them all to be an unsuitable base from an engineering perspective for my purposes.

I’ve also found what I consider fragile support for each as well:

JavaFX: With Oracle taking over Sun, I believe that we have lost a major supporter of quality open source engineering languages and tools.  Oracle’s record of doing what’s right for engineering efficiency and promoting standards even if means potential loss of leverage is not good.  Sun was ok at best but it will in all likelihood just get worse from here.  JavaFX is not as open as Java and is not fully compatible with Java so it almost looks like Sun was trying to fork Java back into a proprietary language before the merger.  Can Sun help Oracle?  In my experience, Macromedia had a great affect on Adobe’s culture in terms of opening up engineering, raising awareness of standards and treating developers as an important constituency.  But at the end of the day, Adobe’s management determined what went down and that did not change after the merger.  I don’t expect Oracle to be more open than Sun or more successful at advancing such a core technology smoothly.  Their business unit managers like to make decisions about technology.  I have not met any Oracle BU heads personally but I feel like I know a couple of them because every meeting I’ve been at with Oracle starts with a discussion of their thoughts.  This is in contrast to Google where it appears like the engineers make decisions about which technology to use for a particular solution.  So I am not optimistic about JavaFX’s long term prospects to solve our core engineering challenges.

Ruby: A dynamic language, without strong typing has very little chance of ever competing on performance with a compiled language.  If you can’t turn “a.b” into “load from offset” and instead need a method call you are sunk from the get go.  I believe a language for writing languages is a great prototyping tool but a poor way to enforce design practices and build a single consistent language adopted by a broad community.   In terms of support, there is no one officially paid to work on the original Ruby runtime that most people use but there are paid projects trying to migrate them to Java and .NET.  Without compatible native plugins though, these efforts are guaranteed to fragment the community and so far have not helped really improve Ruby’s performance or toolability.

Scala: The best attempt yet to make an advanced functional language suitable for the masses but advanced functional languages are not suitable for the masses.  It seems that most of the interesting languages these days are coming from the academic world but I think academics have a different focus than commercial programmers.  Systems tend to be built and maintained by the same person in academia but in the corporate world, code will get handed off and probably to someone with less programming experience than the author.  Scala is also a language for writing languages – again a poor choice for mainstream engineering.

We are all used to free languages but this has left a void of credible language and tools companies.  JetBrains, one of the few has recently open sourced the core of their Java tool suite so that they can compete with Eclipse as a tools ecosystem effectively.   I hope their market is still solid so they can continue to innovate as Eclipse would not be nearly as good without them.

Google shines in this space, continuing to make incredible contributions across the spectrum.  They have yet to show that they are using their leverage unfairly.  You do not see them making corporate bets on any one language – they support Python, Java, C/C++ and have released two languages Simple and Go in the last few months.  Neither of these look particularly strategic to me. Simple is another visual basic-like language designed to help entry-level programmers be more productive but does nothing to improve their workflows with other types of programmers.  Go might be interesting as an alternative to C but like C only targets systems programmers.

I should mention IBM as they have been big supporters of open source projects in the past and I do think they’ve done a lot of good in the industry over the years.  But their challenge is that for any truly horizontal product, there are dozens of competitive efforts internally and so they are not likely to drive change as quickly as would benefit the industry as change for them involves considerable risk and retraining.

Along with researching the industry, I’ve also been building a new platform so the question of how best to market it in today’s climate is something I’ve been doing a lot of thinking about.   All I’m sure of is that it will not be easy.   We developers are tough to sell to.  We have a reluctance for lock-in, desire for all source, and complete control over every link in the supply chain without royalties.  The risk of going overboard of course is that we do not invest directly in those tools and lose out on competitive advantage to tools which improve our productivity.   For now, my project will be independent but I’m interested in any ideas you have for the best way to market platforms in today’s climate.


9 thoughts on “Understanding the Market for Software Platforms

  1. Hi Jeff,

    Nice post about the current situation of some technologies.
    I understand you in some sense. Working with CORBA in the beginning of my career ( around 1998) I had that feeling that whatever program/application I would write, it would run and interoperate with other programs. That was a great feeling. You model the main components of your application in IDL then it’s your choice to implement it in whatever language you are more familiar with.

    Looking back now, it looks like the DSL is what will evolve, generalistic languages require a huge initial investment. A language that will be easy to learn (high level), run across multiple platforms(linux, windows, mac, iphone, android…) , is suitable for enterprise (clustered) and can be used to helps us create 3D applications, that is something I still look forward to see.

    About the best way to market your project, like in any time, a good strategy is that you care, but not too much.

  2. Hello,

    I agree with your opinions about the market of software platforms. But, I don’t belive in the next big language; I think there is a look for each category of problems. In the future, I expect better tools, not better languages; for example, in my “dreams”, a scheduler at process level (instructions are total ordered “threads”).

    Jeff, do you think C++0x will get back what it lost in the last years?

    1. I agree better tools will play a bigger part than languages in improving our productivity going forward. That said, there are a few design patterns I think will move the dial enough to make them worthwhile.

      Good question about the C++ family… ultimately I don’t suspect the number of C++ developers to grow and if anything it may shrink. Security and complexity of the language are the two biggest problems. Performance will always be an issue but I think secure/stable computing will be an even more important thing going forward. That said, C++ developers will probably always make more than other developers on average cause it is the hardest and some problems still require big memory, or “as fast as possible” computing. I don’t suspect java or any other runtime except maybe objective C to take over that any time soon, but as component sets that do stuff like that are imported into the JVM you’ll need to use C++ for a smaller percentage of the problems.

  3. I couldn’t agree more.

    We wholeheartedly adopted your Adobe product in late 2005/early 2006. With passion and exuberance, since then we have based upon it every line of code we’ve written, because it solved so many problems inherent in all other stacks.

    It brought us platform independence on both the client and the server, it brought us an efficient communication protocol, and even a delivery method for the client software. It was also extensible, and flexible. An unsupported feature could be built, and added to the mix.

    To finish a new project, we only had to supply the business logic, and the arrangement of visual components. We were allowed to focus on what was different about a given project, instead of what stays the same each time. It provided a level of productivity which allowed us to be competitive.

    I mention these benefits because they are applicable to your new platform. Reuse of tested components which “learn” from their environment allows the user to function at a high level of abstraction. Meta information, applied to a specific situation is where the component becomes automatically specialized for a given task.

    We’ve been hit hard with the new Adobe pricing because we were putting together an affordable enterprise solution, not a pricey one. We have the “make it up in volume” mentality. To make a solution accessible to the masses seems like the correct market strategy for newcomers and old timers alike. Despite the pricing issues, we are motivated to see it through!

    Thank you so much for sharing your perspective.

  4. Matt, thanks for the feedback, it’s nice to hear from other folks that see how these declarative design patterns can make development and maintenance of sites so much much efficient. The world does not need so many software infrastructure vendors. We just need a few really good, affordable ones to standardize around. When I joined there a few years ago, I believed Adobe could be in that camp but their inability to market affordable server software will be their undoing in this market. Without servers, they cannot have round-trip workflows they need for efficient RIA development. They want to slap server developers in the face while at the same time courting client side developers? Hello, we are the same people. I feel especially pained for the financial burden placed on folks like you who helped them debug the core technology they are now using to inflate the prices of their livecycle enterprise solutions. I doubt they even really care about LCDS standalone sales one bit as it is not even on the main “livecycle” products page. It’s all devoted to their enterprise suite. According to their marketing, livecycle es does not need developers (in reality, you need specially trained developers) so they are trying to put us all out of business while at the same time trying to sell us tools. Not a winning business for them and essentially a train wreck for any developer foolish enough to get too close.

    1. I’ve been gone for a while so not sure how they’re doing now. Back when I was there, it would have done better IMO but the pricing was out of touch with the potential customers. There was no product that would go nicely with flex builder… the free 1 CPU version tended to be ignored by professionals who required clustering and were afraid of being forced to buy the 30K version. It’s a hard technical sale and not well suited to direct sales. Sales did mostly just take orders and there were some OEMs and SIs that helped… really though the users were split into the two camps – the free ones and those who could afford the enterprise version.

      LCDS covered a lot of ground; from reliable realtime to simple declarative ORM based persistence. It combines the two into a single programming model with lots of features you can turn on and off. Some of those really are enterprise features competing with enterprise messaging products out there like lightstreamer, and others in the financial world mostly. For simple persistence there are many alternatives: GraniteDS, Midnightcoders.com, and look at the stuff by Farata systems. Many people just stick with the free simple messaging and remoting in BlazeDS. What makes LCDS 3 special is the combination of a powersoft like toolset and programming model with a well designed, efficient, scalable messaging system. If priced properly, it would have been a win for both developers and the enterprise sales team. Without developers who know how to use all of that stuff and to refine and enhance the tools, solutions built with it will be hard to maintain even for the enterprise customers. Eventually they’ll stop using it and go with the cheaper, more open standards based toolset.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s