Percona has been building and contributing to open-source software since the company was founded, and individually we’ve been doing the same thing for many years. We think it’s a huge value for our customers and the community.
We’re involved in a dozen or so open-source projects, but our three core efforts at the moment are the following:
- Percona patches, which are included in our own MySQL builds and then in OurDelta builds and perhaps others as well
- XtraDB, which is our new high-performance transactional storage engine
- Maatkit, which is a toolkit that provides advanced functionality for MySQL.
We have a team of dedicated MySQL developers working on the server and on ExtraDB, and a dedicated Maatkit developer. Other Percona employees also put significant time into these projects.
Outside observers have commented that our development process doesn’t seem very open-source. That is, we typically just go build the software and then announce it. We don’t involve the community very much in our decisions about what features to include, or how they should get built; and we don’t encourage community contributions directly into our codebase. There’s also some ambiguity about where the money comes from and where it goes. These are all fair points to bring up. Peter, Vadim and I thought we should address them and let everyone know how we really work on these things and what our vision for the future is.
How We Decide On Features
There are at least three sources of input into how we prioritize our work.
- Clear and present needs. If we are crippled day in and day out by the lack of some feature, or the presence of some bug, then we will take the initiative to fix it. The most obvious example is our enhanced slow-logging functionality, which is part of our standard patches. These patches make it possible to learn a great deal about what your server is doing and how to optimize it.
- Customer demands. Customers come to us and ask for the features that they see as clear and present needs; or they come to us with particular performance issues and we analyze it and find that the best solution is to modify the server. Many of the other features in our patch set are in this category, as is a lot of ongoing work that’s not released yet.
- The general public. People comment on our blog posts, or send us email, and so forth. We notice these requests, and if they come up often enough and we think they’ll be useful, they start to stick in our brains and we eventually put them on the TODO list.
The order of priorities is this: paying customers, obvious needs for our consulting practice, and the public wishlist. However, even when work is sponsored by paying customers, we don’t just blindly put the features into our Percona builds. Firstly, customers get to choose whether they want us to do this. Secondly, the builds are designed to meet real-world needs and we are working very carefully to enhance the MySQL server but not end up with a chaotic mix of random stuff.
Some features are also so specific to certain customers that we don’t include them. When we do target an improvement for our builds, it’s a big win for everyone, because we share the development cost with the customer (you can think of it as an open-source discount; it is us contributing towards the public good) and we maintain the feature in subsequent minor versions of the server free of charge. Features that we don’t include in our public patch set don’t get forward-ported for free, so customers pay if they want a version that is ported to a newer server.
In a separate category, there’s also work that we do for customers which is not open-source. We are currently customizing the MySQL server and storage engines for some of Silicon Valley’s most innovative startups. I won’t talk about this any more, but I wanted to mention it for the sake of full disclosure.
How We Support Our Work
This work is usually supported by our customers. (It happens, but not often, that someone just writes new functionality for free in spare time.) However, if the work is open-source, we generally run this as a “less-for-profit” part of our business. I’ll clarify that in a minute.
There are generally two ways that we channel money to features. The first is when a customer says “I want this” and pays us to build it. In these cases, we work on a schedule to deliver the required features. We treat it as developer-for-hire with all the normal expectations of being hired. If we all agree that the result will be open-sourced and the general public will benefit, then we work for a discount. We’re still working for a profit — part of our business is custom MySQL development, after all — but we discount it.
The second way is when people say “here’s some money for the general fund to support your future planned features.” In this case, the money goes into the queue for when someone is available to work on the modifications, and we either pay attention to any preferences the donor has, or we use our judgment as to which enhancements will add the most value to our builds. In practice, this second model has not happened very often; we have used it more for Maatkit and MMM than for server patches. But we hope that will change as more people find out about the possibility for getting their wishes. This model lets us work on features that everyone wants but no one wants to pay for. And in this case we work on a non-profit basis: the developer who works on the features is paid at a special rate for “internal work” (as opposed to working for “external customers”).
You can also think about this as dual sources for the money to support the open-source efforts. One is customers who channel money directly to development. (Thanks!) The other is all of our customers who are indirectly sponsoring the development just by being customers. In other words, if you’re a customer, you’re contributing to open-source software development. (Thanks to you, too!)
If you would like Percona to create custom MySQL features or builds for you, please contact us.
Why We Don’t Grant Commit Rights
As far as we know, no one has ever asked for commit rights to our codebase.
That aside, the Percona contributions to open-source are not a potluck dinner where everyone brings something and puts it on the table. These builds are stamped with our name and are downloadable from our website. They’re a core part of our offering to customers (and non-customers can get them for free, too). If you’re a customer, we stand behind the builds and we support them. The builds power some of the most demanding MySQL installations in the world. All code has to be vetted very carefully, and all features have to deliver a compelling value.
There’s an overarching reason, though. Percona is not the ultimate upstream source. Community contributions ought to be going into the MySQL codebase, not the Percona codebase. Percona’s relationship to the rest of the community is that of a peer, not a parent.
Perhaps some of you are wondering, “why don’t you contribute your patches back to MySQL?”
We do contribute our patches. They are GPL-licensed, which is both generous and fair, and MySQL is free to take them without even asking permission, as long as they abide by the GPL. We welcome everyone to use our patches. OurDelta does so — it is a great example of how the community can aggregate patches and make something greater than the sum of the parts.
When it comes to Sun/MySQL, we actively and openly talk to them about it, and it’s an ongoing conversation. However, there are various reasons why it’s not the ultimate solution for our customers, and their interests come first. For one thing, getting patches accepted is a somewhat laborious process, and we don’t have the resources for it ourselves. We could do it, but we’d have to pass the costs on to our customers.
The second thing is that it takes far too long for patches to actually show up in a GA version of the server. Customers pay us to solve their problems now. This means that one way or another, we’re going to be maintaining the custom patches in our own tree and making our own builds, and there is no sense in doing a halfway job of it. So that removes the motivation for us or our customers to try to get Sun/MySQL to accept the patches — it wouldn’t add any extra value. We’d like Sun/MySQL to accept the patches, but with the customers’ needs solved, the onus is on them to do it, which seems fair — after all, Percona and Percona customers already co-sponsored the development, and Sun/MySQL will be reselling the results.
This is a large and contentious topic that has been discussed at great length elsewhere and is not appropriate to get into here in its full scope. But I’ll point out that some of our patches have been accepted in the past and there are signs that this might continue. We certainly hope so.
How You Can Get Your Wishes To Come True
If you have a particular need for some modification to the MySQL server, XtraDB, Maatkit, our InnoDB data recovery tools, MMM, innotop, or even projects like standard GNU utilities, we’re listening. You can contact us through our website, and offer to sponsor the work. There are also mailing lists for our various projects.
Here’s an example: wouldn’t it be great if there were a built-in type in MySQL that could store date and time with microsecond precision? Lots of people have asked for that. If you all put some money in the hat, it could become a reality.
Where It All Begins
Our open-source efforts flow from our company values. We believe that software and information wants to be free. We also support innovation and entrepreneurship. We like to combine these things when we can! The complexity of our customers’ different situations leads to the need for a lot of flexibility in our approach, which is why there are so many nuances. And we generally try to create incentive towards what we see as the public good. That’s the motivation behind our shared-development-cost model, for example — it creates an incentive for customers to say “yes” when we ask them to agree to open-source the features.
Open-source is not a passing fad. It’s not a marketing gimmick for us. It’s who we are at the core. Funding its development in a fair way is a great balancing act, and finding a balance that seems to make sense for everyone is very rewarding.
Entry posted by Baron Schwartz | No comment
Add to: 
| 
| 
| 
| 