where do we go from here? the future? of open source? and - - PDF document
From: Paul Ramsey <pramsey@cartodb.com> where do we go from here? the future? of open source? and geospatial? This talk is supposed to be about the next ten years, but I felt the topic really called for more question marks. So this is
where do we go from here?
the future?
and geospatial? From: Paul Ramsey <pramsey@cartodb.com>
This talk is supposed to be about the next ten years, but I felt the topic really called for more question marks. So this is ... paul ramsey
2001- 2009-2014 2015-
So, first a little about me, <X> I’m a co-founder and developer of the PostGIS open source spatial database project, <X> For six years, I worked for a professional open source support company, known first as OpenGeo and now Boundless, <X> And I currently work for a software-as-a-service company, CartoDB, that is built on 100% open source infrastructure, And that background definitely informs this talk, as you’ll see… paul ramsey
I have been allowed to speak at FOSS4G plenary sessions five times now, every two years, and from 2009 on my topic has always seemed to come back around to economics. The economics of open source software development. economics
are CRAZY
Which isn't surprising, it’s an interesting topic, because the economics don't make any sense. When I grow up, I’m going to give software away for free! Sport, you’re out of your mind.
The economics don't make sense… <X> when you try to explain them to your family. Sir, we should make the new software
Do I look stupid to you? You’re out of your mind.
The economics don't make sense… when you try to explain them to your boss. Could go rock climbing... Could close tickets... I’m out of my mind...
The economics don't even make sense… <X> when you try to explain them to your self. here i am
Now, obviously I am here, 8000 kilometers from home, speaking to you, well-fed and wearing, pretty nice shoes, so the economics can't be all *that* bad. the economics of open source are bad,
But they are bad enough. Bad enough to be worth mentioning. (support)
ha ha, a funny
https://twitter.com/xof/status/622113231218192384 "Hi I'm an engineer at a well-funded company and we need this feature can someone implement it for free?" There's a reason all the open source developers laugh and nod and re-tweet this classic: it's all too true. ha ha, a funny
https://twitter.com/howardbutler/status/348168348795797504 My favourites in the genre combine a complaint about a missing feature with a threat to use some other software. Or there's this variant, ha ha, a funny
https://twitter.com/jordansissel/status/265528663339069440 "Hi, I'm a consultant who was hired to do this thing I cannot do. You do it for me. URGENT." Whether the request comes from someone at an outsourcing bodyshop, incredibly good at creating wealth and incredibly bad at capturing it
The open source software model is incredibly good at creating wealth and incredibly bad at <X> capturing it. The canonical proof for this is the Heartbleed episode. heartbleed
For those of you who missed out, the synopsis is this: In late December of 2011... december, 2011
..., a bug was accidentally introduced into the OpenSSL network encryption library. OpenSSL is at the heart of almost every secure connection made on the internet. $1,000,000,000s
To say that OpenSSL is responsible for the security of billions of dollars of commerce every day would not overstate its important to the global internet economy. The bug, which was nicknamed "Heartbleed", was nothing more than a single-line bounds-checking error. april, 2014
Heartbleed was discovered and announced in April of 2014, leading to a world-wide rush to patch the vulnerability on every server on the internet. Millions of dollars were spent, retroactively, to paper over this small mistake in software. So, how could a simple error like this pass code review and get added to a library that protects billions of dollars in global commerce? developer
feature development
“The mystery is not that a few overworked volunteers missed this bug; the mystery is why it hasn't happened more often.”
President, OpenSSL Foundation
"The mystery is not that a few overworked volunteers missed this bug; the mystery is why it hasn't happened more often." OpenSSL generates billions of dollars in value, but failed to capture even a few hundred thousand dollars to maintain a staff of core maintainers. And because of that, millions of dollars were spent in the end, remediating a bug that slipped through. gnupg ntpd
After Heartbleed, some of the major internet companies, like Google, Facebook and Amazon, created a new fund for maintaining key internet infrastructure, and OpenSSL was among the first reciptients of funding. So, problem solved? enlightened self-interest
Google and company are displaying enlightened self-interest; they are supporting infrastructure that is critical to their commercial mission. The trouble with enlightened self-interest, though, is that it still requires enlightenment. enlightened self-interest
And it's not permanent. If, in five years, Google decides to save a few dollars, self-interest
“let someone else take care of that...” “maybe they will fix it in the next release...” “facebook is funding maintenance? suckers...” “why don’t you do a better job with quality control...” “that’s not in our core mission...”
and leave the funding of the critical infrastructure project to Facebook and Amazon, there's nothing stopping them. And Google will still receive all the benefits of Facebook and Amazon's investment: after all, OpenSSL is open source! Letting others pay, while you still play, is called being a "free rider". free riders cooperators
public goods (aka OSS)
benefits contributions
The nice thing about open source, and digital media in general, is that there's nothing wrong with free riders, in that they don't *cost* anything. They don't add to the burden of the work of the project, as long as they don't ask questions and just quietly use the software. free riders
cooperators
public goods (aka OSS)
benefits contributions
The danger of free riders is when the population of software users is so big, and so dominated by free riders that the societal impact and importance of the software dwarfs the resources that are being devoted to its maintenance and longevity. This is what happened to OpenSSL with Heartbleed, and it happens to projects in the open source geospatial world too. (JTS, GEOS and PROJ would be the obvious examples.) here i am
But wait! Again, here I am, standing 8000 km from home, wearing... passably nice shoes and looking well fed, the core committer of an open source project. What do **I** have to complain about? me? no complaints
Me? Nothing. Nothing at all. But I'm a corner case. There are hundreds of developers in the open source geospatial community contributing most your job is to make the open source software better!!!
BUT there are only a handful like me who are employed with a job description that explicitly includes the goal of "making the open source software better". Most open source geospatial contributors are employed at jobs where building open source is a side effect of their real responsibilities. the economics of open source are bad, except where they are good;
And that's actually good news, because the economics of open source are terrible, <X> except where they are good, and that's in building systems *using* open source software. use value sale value vs
http://www.catb.org/esr/writings/magic- cauldron/magic-cauldron.html
Eric Raymond touches on this aspect of open source economics in his essay “The Magic Cauldron” when he talks about the distinction between the "use value" and the "sale value" of software. use value sale value
your parcel management system your financial accounting system your harvest forecasting system your revenue collection system your patient intake system
We get all excited about the "sale value" of software, because it's the kind of software that gets advertised and that our departments go out and "buy": Oracle and Microsoft and ESRI software. But despite the huge mindshare of these companies, making their revenue from the sale value of software, most software written is never sold. <X> Most software is built and used, it has a use value, but it has no sale value. Financial services back-office systems, government renevue systems, cadastral management systems, they all have tremendous value in their use, but they weren't BOUGHT, they were CONSTRUCTED. software for sale
And espite our fixation software for use
software for sale
it is on purchasing services to build software for use. IT Software
>$10M
IT Services
>>$270M
In my province, British Columbia, Canada, the government standardized *all* databases on Oracle back in the 90s, so we have quite a fleet of licenses for Oracle. That cost the government somewhere around $10M last year, which is a lot of money, but compare that with the services. <X> Between HP, IBM and CGI, the government spent over $273 million on services. And that's not all the service providers by a long stretch. GIS Software
~$1M
GIS Services
>$10M
Similarly, the British Columbia government spends about $1M on Esri software, <X> but over $10M on various GIS services companies. As a single global software vendor, Esri is very large, but as a component of the overall spatial IT market, it's quite small. Software
OS Database Renderer Cache Map Windows Oracle ArcGIS ArcGIS Google Linux PostGIS *Server *Cache *Layers
Because systems are built from parts, and so as long as the parts are functionally interchangeable <X> it doesn't matter if you use open source or proprietary. services companies will improve open source to meet client needs (and not tell the client) because the client cares about the system, not the parts
If you track back into the new features added to various open source projects, it's not uncommon to find a larger IT systems project in the background. Just from my personal experience, I know the following open source development was done to meet service contracts of the stealth open source in
big investments big company
small investments small company
There’s an irony on the services side of open source investment in that the larger the system, and the larger the vendor, the less likely the vendor is to invest in new open source features or development. This IS changing over time, as customers get more comfortable with open source components, so vendors can feel comfortable bidding them as part of a solution. enlightened self-interest
It SHOULD be happening much faster, since bidding an open source solution as a service company isn't "enlightened self-interest", it's just plain old, standard issue, greedy "self-interest". less software more service
self-interest
Every dollar you avoid sending to a proprietary software vendor is an extra dollar you can slurp up in services. And If that were the whole story, I'd feel pretty good about the long term, the economics of open source are bad, except where they are good; but the trends are bad,
yes, the economics of open source core development are bad, and it's hard to make a go of it as a pure-play developer, but <X> the market for systems integration using open source is only growing, and even a tiny fraction of those dollars can keep open source projects, if not healthy, at least alive, but... <X> The only constant in the IT business is change, and the long-term trends are a bit scary for open source. There are big changes afoot: mobile! mobile! mobile!
* So, we know mobile is huge, says these these don’t have
and we know that aspects of mobile software, like the distribution via hostile app stores that disallow some common licenses, <X> and the disconnect between the usage context and the development context, are all worrying trends, but I think all those problems can be overcome. What I'm really worried about is the infrastructure serving those mobile devices. the-future-as-a-service
I'm worried about clouds. * IT is turning into a utility. In the future, everything, EVERYTHING will be "as a service", and that has serious implications for open source communities and development. 1900s electricity trends 2000s computing trends
just like
Nicolas Carr has written a great book that compares the move to cloud computing architectures to the rise of utility grade electricity, "The Big Switch". We're in the midst of a "Big Switch" in IT, and the parallels with the big switch a century ago are very strong. We've forgotten that there was a transitional period in the history of electrification, after the commercial success of electricity, but before it faded into the background of our lives. power millwork
Prior to electrification, <X> factories were run with "millwork", with big systems of belts and gears, powered by <X> on-site steam engines or waterworks. Based on this pattern, industrial organizations were used to, and comfortable with, generating their own power, and after electrification they kept on doing so. belt-driven tools electric tools
Factory owners though electrification was great, because it removed all the dangerous and inefficient belts and gears from the factory floor <X> and freed the tools from the belts, so they could be freely moved around and reconfigured for different jobs. Electrifying your factory was a HUGE competitive advantage. The trouble was, in the early days, if you wanted electricity, you had to purchase your own generator. generation
You ordered a generating package and transformers and wires, and you set them up in your works, <X> with the help of a big staff of technicians and skilled tradespeople to keep the big complex thing running. pearl street generator 1882
Now, Thomas Edison set up his first large scale demonstration generating station by 1900
50,000
generating stations
By 1900, 18 years later, the US Census recorded over 50,000 site-specific generating stations in the USA. Each of those stations would have had dedicated staff and expertise managing them and keeping them running. Does anything sound familiar? the cost of doing business
Electricity was a HUGE competitive advantage for factories, so they were willing to invest in onsite infrastructure and expertise. For a time, having an onsite generating station was just, the cost, of doing, business. the cost of doing business
Similarly, information technology has quickly gone from a nice-to-have to a have-to-have and organizations of all sizes have invested in the overhead of in-house infrastructure and technical teams. Some of those teams are even using open source to effectively deliver their services. utility
The economics of generation and economies of scale meant that, as electric utilities grew utility
they could eventually provide power far more cheaply than on-site generators. utility
More sophisticated grid technology, meant they could provide greater reliability using multiple generating stations. utility
And so, the factories began shutting down their old generators and taking power directly from the grid. Though some stubborn ones held out longer, preferring the local control and independence utility
eventually, they all gave in. We're seeing the same transition in IT. It's not as abrupt as flipping the switch on grid electricity, because of the nuance of information technology, but it's just as certain, because it is being driven by economic forces. are your
efficient than... ?
is your email more efficient than... ?
up our data center...
There's no doubt that established organizations don't change very fast. They have history and habits and internal vested interests. Don't judge trends or change by looking at OLD organizations. Look at NEW organizations. what would a brand new company do?
Does a NEW company set up their own data center? Does a NEW company set up their own email server? Do they run their own accounting software, timesheet tracking, calendar system, CRM database, hiring pipeline? OldCo
in house data center Amazon Web Services in house e-mail Google Apps accounting system QuickBooks Online custom timesheet Harvest in house calendar Google Apps in house CRM Salesforce.com recruiting system Resumator
I note all those functional categories, because back when I set up my first business, in 1997, I installed and managed *all* those information management systems on-site. At the two relatively brand new companies I have worked for since 2009, all those functions and more are consumed as services over the web. “well, duh...”
Well, so what, you don't need me to tell you IT is moving to the cloud and to an "as a service" model. What I want to remind you of is the lesson from history. utility
All those little generating stations, when they were shut down, where did the technicians and the skilled trades people go? utility
Some went to the utilities, but not all of them: the whole point of the utilities was to run at a larger scale, more efficiently, with bigger generators and FEWER workers. utility
The centralization of IT into cloud services means a centralization of skills, and a reduction in the number of people who can apply those skills. Each centralization and commoditization of IT is going to shut doors and shrink the base of available expertise. Not that this is all bad... the economics of open source are bad, except where they are good; but the trends are bad, except where they are good.
Yes, the economics of core open source development are pretty bad, except that the core development <X> enables a huge and profitable services ecosystem stringing parts together into systems, <X> except that systems are increasingly being hoovered up into the cloud and more and more functionality is being offered by a small collection of "as a service” companies <X> but... utility computing companies invest in open source
By and large utility computing providers have been very good at recognizing that there is great value in investments in open source. utility computing companies invest in open source because
is their critical infrastructure
These companies use open source a lot, but they don't get revenue from it, it's infrastructure. gnupg ntpd
So we get the Google/Amazon/Facebook investments in OpenSSL, and similar support of the Linux Foundation. We get open source software released directly from those companies, like Cassandra from Facebook, postgis mapnik
leaflet turf torque
And in our own field, we get online companies like MapBox, CartoDB and Mapzen working entirely in the clear, building services on open source components, but deriving REVENUE from online SERVICES, not software sales or direct software support. the big squeeze
fewer contributors? but with more resources?
So the consolidation effect of the cloud is working both FOR and AGAINST open source. It's narrowing the population of potential contributors over the long run, but in the present it's also providing some otherwise unparalleled opportunities for core developers to do paid work on their projects. lane van rossum agafonkin ramsey???
For example, <X> Tom Lane working on PostgreSQL at Saleforce.com. <X> Guido von Rossum working on Python at Google and now Dropbox. <X> Vlad Agafonkin working on Leaflet at MapBox. Or *cough* me, (me?) <X> working on PostGIS at CartoDB. (See what I did there?) the economics of open source are bad, except where they are good; but the trends are bad, except where they are good.
So, Yes, the economics of core open source development are pretty bad, <X> except that there is a huge and profitable services ecosystem stringing open source parts together into systems, <X> but unfortunately the trend to cloud "as a services" is reducing the scope of systems work <X> except those "as a service" companies are actually being pretty good (so far) at supporting the open source they use under the hood First they ignore you, then they laugh at you, then they fight you, then you win.
you are here
OK, So, that’s a lot of dire stuff, really, really depressing stuff, so let’s remind ourselves of where we are. Over the past 20 years, open source has gone from the fringes to the mainstream of IT. It was a slow transition, and it’s still going on, but, we, won. We won. So the present is really, really good. and then what happens?
And then what happens? The only constant is change, lots of change, And the move to open source is just one trend among many. In the long run...
cloud services ready-to-use less fun tinkering
Over the next 10-20 years, I expect <X> to see the cloud take up more and more of the oxygen in the IT world, I expect <X> to see customers to expect their services to arrive "pret a porter", ready to user, and I expect to see customers have a <X> diminishing tolerance for customization and fiddling of the sort that personally I enjoy. And that feels like bad news for me, as a tinkerer, though it’s probably good news for the customers. But that's in the long run, potentially the very long run. And in the long run, in the very long run, In the long run, we’re all dead.
in the short run...
In the short run, in the here and now, keeping the ethos of open source alive and well is the most important thing, because that ethos will influence the path of all the change to come. and that ethos is not about money or economics, collaborating sharing teaching enabling
it’s about working collaboratively it’s about <X> sharing knowledge, not hoarding it it’s about <X> teaching others what we’ve learned and it’s about <X> giving them the tools to build something we could never have imagined... .. one of my favourite quotes is the well-known one from Isaac Newton “If I have seen further, it is by standing on the shoulders
you are all giants
You’re all giants. Be proud of what you do and keep on doing it. build great things!
the future is open source geospatial! To: Paul Ramsey <pramsey@cartodb.com>
Thank you.