Sunday, December 3, 2023
Major Price Cuts: Deepnote Versus Cocalc --- Compute Server Pricing
Major Price Cuts: Deepnote Versus Cocalc
Deepnote is one of CoCalc's direct competitors. Today (November 30, 2023) they announced a major price cut on their pay-as-you-go rates:
"As you may have already heard, starting December 1, we're slashing the pay-as-you-go rates across all our machines – making them more budget-friendly without any hidden terms."
image
At CoCalc, we recently finally launched pay as you go machines, which was one of our main development priorities for 2023. These are fully integrated with CoCalc, and were a huge amount of work to bring to market. I was terrified that Deepnote's major price cuts would make Deepnote a much better deal than CoCalc.
Here is how the Deepnote and CoCalc pricing compares:
| Deepnote's New Price | CoCalc Standard | CoCalc Spot | |
|---|---|---|---|
| 64GB RAM, 16vCPU | 1ドル.54 | 0ドル.59 | 0ドル.12 |
| 128GB RAM, 16vCPU (32 CPU on cocalc) | 2ドル.02 | 1ドル.17 | 0ドル.23 |
| K80 GPU (newer L4 GPU on cocalc) | 2ドル.02 | 0ドル.93 | 0ドル.30 |
Conclusion: CoCalc's prices are still highly competitive, even in light of Deepnote's major price cuts.
Also, spot instances do work very well for many applications. For more details and how to get these prices at https://cocalc.com, read the rest of this post.
CAVEAT: comparing RAM and vCPU is not necessarily easy. Maybe I'm completely wrong.
More Details
I don't know exactly what Deepnote means by the above machine specs. However, according to my benchmarks, one of the very best machines we offer via Google Cloud is the AMD EPYC Milan family. Their single core performance is excellent, and a vCPU is equivalent to an entire core, which makes them up to twice as fast as lot of "vCPU" options out there. We offer both spot instances and standard instances.
Performance: 16 vCPU and 64GB RAM
Our best pricing on an AMD EPYC with 64GB RAM and 16 cores is 0ドル.59/hour for standard instances.
image
By selecting a region in Europe, the cost is only 0ドル.12/hour for a spot instance. Spot instances may stop or not be available, but our stats so far show they often work well for days to weeks, perhaps because Google has built out such massive CPU capacity:
image
In CoCalc the region where the machine is located is transparent, so you can take advantage of the best prices in the world.
High Memory: 16 vCPU and 32GB RAM
Our analogue of "High memory" above is a t2d-standard-32 with 32 cores, 128B of RAM, and it costs 1ドル.17/hour for a standard instance, or 0ドル.23/hour for a spot instance.
image
Again, the best price on spot instances is in a different region than for standard:
image
GPU
Deep note offers a K80 GPU for 1ドル.80/hour. We do not offer K80's on CoCalc since they are so old, but we have L4's that have the same 24GB of RAM and are a much newer architecture. Our GPU price is 0ドル.93/hour for standard instances, and 0ドル.30/hour for spot instances:
image
Conclusion: CoCalc's new prices are still competitive. Yeah.
Happy Holidays! 🎄
Thursday, May 9, 2019
Should I Resign from My Full Professor Job to Work Fulltime on Cocalc?
Nearly 3 years ago, I gave a talk at a Harvard mathematics conference announcing that “I am leaving academia to build a company”. What I really did is go on unpaid leave for three years from my tenured Full Professor position. No further extensions of that leave is possible, so I finally have to decide whether or not to go back to academia or resign.
How did I get here?
Nearly two decades ago, as a recently minted Berkeley math Ph.D., I was hired as a non-tenure-track faculty member in the mathematics department at Harvard. I spent five years at Harvard, then I applied for jobs, and accepted a tenured Associate Professor position in the mathematics department at UC San Diego. The mathematics community was very supportive of my number theory research; I skipped tenure track, and landed a tier-1 tenured position by the time I was 30 years old. In 2006, I moved from UCSD to a tenured Associate Professor position at the University of Washington (UW) mathematics department, primarily because my wife was a graduate student there, UW has strong research in number theory and algebraic geometry, and they have a good culture supporting undergraduate research.
Before I left Harvard, I started the SageMath open source software project, initially with the longterm goal of creating a free open source viable alternative to Mathematica, Maple, Matlab and Magma. As a result, in addition to publishing dozens of research mathematics papers and some books, I also started spending a lot of my time writing software, and organizing Sage Days workshops.
Recruiting at UW Mathematics
At UW, I recruited an amazing team of undergraduates and grad students who had a major impact on the development of Sage. I was blown away by the quality of the students (both undergrad and grad) that I was able to get involved in Sage development. I fully expected that in the next few years I would have the resources to hire some of these students to work fulltime on Sage. They had written the first versions of much of the core functionality of Sage (e.g., graph theory, symbolic calculus, matrices, and much more).
I was surprised when my application for Full Professor at UW was delayed for one year because – I was told – I wasn’t publishing enough research papers. This was because I was working very hard on building Sage, which was going extremely well at the time. I took the feedback seriously, and put more time into traditional research and publishing; this was the first time in my life that I did research mathematics for reasons other than just because I loved doing it.
I tried very hard to hire Bill Hart as a tenure-track faculty member at UW. However, I was told that his publication count was “a bit light”, and I did not succeed at hiring him. If you printed out the source code of software he has written, it would be a tall stack of paper. In any case, I totally failed at the politics needed to make his case and was left dispirited, realizing my personal shortcomings at department politics meant I probably could not hire the sort of colleagues I desperately needed.
UW was also very supportive of me teaching an undergrad course on open source math software (it evolved into this). I taught a similar course at the graduate level once, and it went extremely well, and was in my mind the best course I ever taught at UW. I was extremely surprised when my application to teach that grad course again was denied, and I was told that grad students should just go to my undergraduate course. I thought, “this is really strange”, instead of lobbying to teach the course and better presenting my case.
To be clear, I do not mean to criticize the mathematics department. The UW math department has thought very hard and systematically about their priorities and how they fit into UW. They are a traditional pure mathematics departments that is generally ranked around 25 in the country, with a particular set of strengths. There is a separate applied math department on campus, several stats departments, and a massive School of Computer Science. Maybe I was in the wrong place to try to hire somebody whose main qualification is being world class at writing mathematical software. This blog post is about the question of whether the UW math department is the right place for me or not.
Outside Grant Support?
My number theory research received incredible support from the NSF, with me being the PI on six NSF grants. Also, Magma (which is similar to Sage, but closed source) had managed to find sufficient government funding, so I remained optimistic. Maybe I could fund people to build Sage via grants, and even start an institute! I applied for grants to support work on SageMath at a larger scale, and had some initial success (half of a postdoc, and some workshops, etc.).
Why is grant funding so important for Sage? The goal of the SageMath project is to create free open source software that is a viable alternative to Mathematica, Maple, Matlab, and Magma – software produced by companies with a combined thousands of fulltime employees. Though initial progress was encouraging, it was clear that I desperately needed significant money to genuinely compete. For example, one Sage developer had a fantastic Sage development project he wanted about 20K to work fulltime on during a summer, and I could not find the money; as a result he quit working on Sage. This project involved implementing some deep algorithms that are needed to more directly compete with Mathematica for solving symbolic inequalities. This sort of thing happened over and over again, and it began to really frustrate me. I could get plenty of funding for 1-week workshops (just travel expenses – everybody works for free), but there’s only so much you can do at such sprints.
I kept hearing that there would be a big one-in-10-years NSF institutes competition sometime in the “next year or two”. People hinted to me that this would be a good thing to watch out for, and I dreamed that I could found such an institute, with the mission to make it so the mathematics community finally owned the deep software on which teaching and research are based. This institute would bring the same openness and robustness to computational mathematics that rigorous proof had brought to mathematics itself a century earlier.
Alas, this did not happen. I remember the moment I found out about the actual NSF institutes competition. Joe Silverman was standing behind me at a coffee break at The Arizona Winter School 2010 telling people about how his proposal for ICERM had just won the NSF institutes competition. I spun around and congratulated him as I listened to how much work it was to put together the application during the last year; internally, my heart sunk. Not only did I not win, I didn’t even know the competition had happened! I guess I was too busy working on Sage. In any case, my fantasy of creating an NSF-funded institute died at that moment. Of course, ICERM has turned out to be a fantastic institute, and it has hosted several workshops that support the development of open source math software.
Around this time, I also started having my grant proposals denied for reasons I do not understand. This was confusing to me, after having received so many NSF grants before. In 2012, the Simons Foundation put out a call for something that potentially addressed what I had hoped to accomplish via an NSF-funded institute. I was very excited again, but that did not turn out as I had hoped. So next I tried something I never thought I would ever do in a million years…
Commercialization at UW
For various reasons, I failed to get the NSF or other foundations to fund Sage at the level I needed, so in 2013, I decided to try to sell a commercial product, and use the profits to fund Sage development. I first tried to do this at University of Washington, by working with the commercialization office (C4C) to sell access to Sage online. As long as the business and product were merely abstract ideas (e.g., let’s make up a name and trademark it! let’s write some terms of service!) things went fine. However, when things became much more concrete, working with C4C got strange and frustrating for me. I was clearly missing something.
For example, the first thing C4C told me on the very first day we sat down together was they would not work with me if I made the software I wrote for this open source, and that the university would own the software. Given there was no software at all yet, and I imagined I would just whip out a quick modern web-based frontend to Sage and make boatloads of money that would go straight into a UW account to be used to fund Sage, this seemed fine to me. However, I had a nagging feeling that a pure closed-source approach to this problem was impossible, and not having that flexibility would come back to haunt me.
Naively optimistic, I found myself working fulltime at UW and at the same time trying to get a sophisticated web application off the ground by myself, with many important early users depending on it for their classes. This was stressful and took an enormous amount of time and attention. I felt like I was just part of the software, often getting warnings that things were broken or breaking, and manually fixing them. The toil was high, and only got worse as more people used the software. I would get woken up all night. I couldn’t travel since things were constantly breaking.
Every time I fought through some really difficult problem with the web application instead of just giving up, I came out far more determined not to quit.
The web application described above evolved over 6 years into what is now https://CoCalc.com; the functionality was pretty similar from day 1, but quality and scalability have come a long ways. CoCalc lets you collaboratively use LaTeX, Sage, Terminals, Jupyter Notebooks, etc., for teaching and research.
In 2014, I went on sabbatical and worked fulltime developing this web application and the feedback loop I described above only grew more intense: fix things, fight through difficult problems, be even more determined not to give up. Fortunately, I had some leftover NSF grant funds, and was able to use them to hire several students to help with development. I failed to find students who I could hire to do the backend work (and be available any time day or night), which meant that much of the stress of keeping the site running continued to fall squarely on my shoulders. And as the site grew in popularity (and functionality), the stress from it got worse.
My Sabbatical ended, and I was required to return to UW fulltime for one year, or return all the money I was paid during my sabbatical. So far, CoCalc had grown in popularity, but I had not been allowed by the “commercialization office” to actually commercialize it, so it was still a free site.
I taught at UW at the same time as being the main person trying to run this very complicated and painful production web application. Based on user feedback, I was also highly motivated to improve CoCalc. I would typically sleep a few hours, get up at 3am and write code until 8am, then prepare to teach, hope not to have any site issues right before class, and so on. One day CoCalc got hit by a massive DDoS attack minutes before a class I was teaching, while I was talking with a prospective donor to the math department.
I am the sort of person who does well focusing on exactly one thing at a time. Given the chance to fully focus on one thing for extended periods of time, I sometimes even do things that really matter and have an impact. I am not great at doing many different things at once.
In the meantime, Sage itself was growing and receiving funding, though this had nothing to do with me. For example, Gregg Musiker was putting together a big program at IMA, in the form of a ton of Sage Days workshops. Also, the huge ODK project, which was a European Union grant proposal to support open source math software would be fully funded. And closer to home, Moore and Sloane funded a major new initiative that could potentially have also supported work on Sage. I was invited to go to workshops and events involving these and other grants, but often I either said no or canceled at the last minute due to the toil needed just to keep CoCalc running. Also, I believed if I could start charging customers, then I would have a lot more money, and could hire more help.
I met with more senior people at UW’s C4C to finally actually charge people to use CoCalc. They wanted me to do some integration with their license management system, and sell “express” software licenses. It didn’t make any sense to me, and we went around in circles. I then asked about actually starting a separate company (a legal entity) that the university would have some ownership in, so that the company could take payments, etc. This is when things got really weird. They would not talk with me about creating the company due to “conflict of interest”.
I searched for other UW faculty that had commercialized remotely similar products, and found one. He told me how it went, and said it was the worst experience of his life. UW owned 50% of the company, and all of the software of the company, which they licensed under onerous terms. They refused to negotiate anything with him, instead requiring his spinoff company to hire an outside negotiator. As a result of all this, I educated myself as much as possible about relevant rules and laws, and consulted with a lawyer.
It turns out that the NSF grants I used to fund work on CoCalc explicitly stipulated that code funded by those grants had to be GPL licensed. This meant all the code for CoCalc had to be open sourced. Later the university even agreed in writing to release a snapshot of all the CoCalc code under the BSD license, and I haven’t been paid a penny by UW since the date of that release, so there is no possible claim that the company can’t use the code.
Building a company
A colleague of mine from when I was at Harvard was in town for a day, and we met for coffee. He expected we would talk about Sage and number theory, but instead I told him about CoCalc and my attempts at commercialization and starting a company. He immediately suggested a solution to my problems, which was to talk with a friend of his who had both extensive experience working with companies and deep connections with mathematics. I was confident that in the worst case I could quit my job at UW and rewrite all the software from scratch, so I took him up on the offer.
In 2015 I formed a corporation, and received some outside investment, and used that (and dramatically cutting my already-small academic income) to “leave academia”. More precisely, in 2016 (after working fulltime for a year at UW), I finally went on 100% unpaid leave from UW in order to completely focus on CoCalc development and getting a business off the ground. Also, there was no good reason to quit a tenured Full Professor job when you can go on leave; also CoCalc supports teaching in math departments, so it is closely related to my academic job. The only academic responsibilities I had were to my two Ph.D. students, who I meet with one-on-one at least once a week. At the end of two years, I requested a third year of unpaid leave, which UW granted (this is not routine). Throughout all this, the UW mathematics department was very supportive.
During these three years on unpaid leave, I’ve hired three other people who work fulltime on CoCalc. Together we have massively improved the software, and built a business with thousands of paying customers. The company is still not profitable, though the future is clearly very bright if we continue what we are currently doing. CoCalc has become a platform that an increasing number of scalable products (such as this) are being built with, and there is enormous growth potential in the next year.
At this point, it rightfully appears to the community that I have left SageMath development to focus fulltime on building CoCalc as an independent business. Indeed, I do not spend any significant time contributing to Sage, and I even switched to getting daily digests of the sage-devel mailing list.
On the other hand, as mentioned above, CoCalc is going well by many metrics (in terms of quality, feature development, customer love, market position, etc.). Most importantly, me and the other three people who work fulltime on CoCalc really, really love this job, and the potential to have a significant impact. I still don’t know if CoCalc will ever be wildly profitable and massively fund Sage development. If I were to obsess over only that goal, I would have to quit working on CoCalc (since it is taking way too long) and pursue other opportunities for funding Sage.
In retrospect, my idea from 7 years ago to start a web-based software company from scratch and build it into a successful profitable business has so far completely failed to fund Sage.
It would be far easier to work fulltime writing grants to foundations, and leveraging the acknowledged success of Sage so far. I made the wrong move, given my original goal. The surprise is that I really enjoy what I’m doing right now!
My unpaid leave is up – what am I going to do?
My third year of unpaid leave from UW is up. I have to decide whether to return to UW or resign. If I return, it turns out that I would have to have at least a 50% appointment. I currently have 50% of one year of teaching in “credits”, which means I wouldn’t be required to teach for the first year I go back as a 50% appointment. Moreover, the current department chair (John Palmieri) understands and appreciates Sage – he is among the top 10 all time contributors to the source code of Sage!
I have decided to resign. I’m worried about issues of intellectual property; it would be extremely unfair to my employees, investors and customers if I took a 50% UW position, and then later got sued by UW as a result. Having a 50% paid appointment at UW subjects one to a lot of legal jeopardy, which is precisely why I have been on 100% unpaid leave for the last three years. But more importantly, I feel very good about continuing to focus 100% on the development of CoCalc, which is going to have an incredible year going forward. I genuinely love building this (non-VC funded) company, and feel very good about it.
Monday, January 1, 2018
Low latency local CoCalc and SageMath on the Google Pixelbook: playing with Crouton, Gallium OS, Rkt, Docker
I just got CoCalc fully working locally on my Google Pixelbook Chromebook! I want this, since (1) I was inspired by a recent blog post about computer latency, and (2) I'm going to be traveling a lot next week (the JMM in San Diego -- come see me at the Sage booth), and may have times without Internet during which I want to work on CoCalc's development.
I first tried Termux, which is a "Linux in Android" userland that runs on the Pixelbook (via Android), but there were way, way too many problems for CoCalc, which is a very complicated application, so this was out. The only option was to enable ChromeOS dev mode.
I next considered partitioning the hard drive, installing Linux natively (in addition to ChromeOS), and dual booting. However, it seems the canonical option is Gallium OS and it nobody has got that to work with Pixelbook yet (?). In fact, it appears that Gallium OS development made have stopped a year ago (?). Bummer. So I gave up on that approach...
The next option was to try Crouton + Docker, since we have a CoCalc Docker image. Unfortunately, it seems currently impossible to use Docker with the standard ChromeOS kernel. The next thing I considered was to use Crouton + Rkt, since there are blog posts claiming Rkt can run vanilla Docker containers on Crouton.
I setup Crouton, installed the cli-extra chroot, and easily installed Rkt. I learned how Rkt is different than Docker, and tried a bunch of simple standard Docker containers, which worked. However, when I tried running the (huge) CoCalc Docker container, I hit major performance issues, and things broke down. If I had the 16GB Chromebook and more patience, maybe this would have worked. But with only 8GB RAM, it really wasn't feasible.
The next idea was to just use Crouton Linux directly (so no containers), and fix whatever issues arose. I did this, and it worked well, with CoCalc giving me a very nice local browser-based interface to my Crouton environment. Also, since we've spent so much time optimizing CoCalc to be fast over the web, it feels REALLY fast when used locally. I made some changes to the CoCalc sources and added a directory, to hopefully make this easier if anybody else tries. This is definitely not a 1-click solution.
Finally, for SageMath I first tried the Ubuntu PPA, but realized it is hopelessly out of date. I then downloaded and extracted the Ubuntu 16.04 binary and it worked fine. Of course, I'm also building Sage from source (I'm the founder of SageMath after all), but that takes a long time...
Anyway, Crouton works really, really well on the Pixelbook, especially if you do not need to run Docker containers.
I first tried Termux, which is a "Linux in Android" userland that runs on the Pixelbook (via Android), but there were way, way too many problems for CoCalc, which is a very complicated application, so this was out. The only option was to enable ChromeOS dev mode.
I next considered partitioning the hard drive, installing Linux natively (in addition to ChromeOS), and dual booting. However, it seems the canonical option is Gallium OS and it nobody has got that to work with Pixelbook yet (?). In fact, it appears that Gallium OS development made have stopped a year ago (?). Bummer. So I gave up on that approach...
The next option was to try Crouton + Docker, since we have a CoCalc Docker image. Unfortunately, it seems currently impossible to use Docker with the standard ChromeOS kernel. The next thing I considered was to use Crouton + Rkt, since there are blog posts claiming Rkt can run vanilla Docker containers on Crouton.
I setup Crouton, installed the cli-extra chroot, and easily installed Rkt. I learned how Rkt is different than Docker, and tried a bunch of simple standard Docker containers, which worked. However, when I tried running the (huge) CoCalc Docker container, I hit major performance issues, and things broke down. If I had the 16GB Chromebook and more patience, maybe this would have worked. But with only 8GB RAM, it really wasn't feasible.
The next idea was to just use Crouton Linux directly (so no containers), and fix whatever issues arose. I did this, and it worked well, with CoCalc giving me a very nice local browser-based interface to my Crouton environment. Also, since we've spent so much time optimizing CoCalc to be fast over the web, it feels REALLY fast when used locally. I made some changes to the CoCalc sources and added a directory, to hopefully make this easier if anybody else tries. This is definitely not a 1-click solution.
Finally, for SageMath I first tried the Ubuntu PPA, but realized it is hopelessly out of date. I then downloaded and extracted the Ubuntu 16.04 binary and it worked fine. Of course, I'm also building Sage from source (I'm the founder of SageMath after all), but that takes a long time...
Monday, October 10, 2016
RethinkDB must relicense NOW
What is RethinkDB?
UPDATE: Several months after I wrote this post, RethinkDB was relicensed. For the CoCalc project, it was too late, and by then we had already switched to PostgreSQL.
Three Types of Open Source Projects
There are many types of open source projects. RethinkDB was the type of open source project where most work on RethinkDB has been fulltime focused work, done by employees of the RethinkDB company. RethinkDB is licensed under the AGPL, but the company promised to make the software available to customers under other licenses.Academia: I started the SageMath open source math software project in 2005, which has over 500 contributors, and a relatively healthy volunteer ecosystem, with about hundred contributors to each release, and many releases each year. These are mostly volunteer contributions by academics: usually grad students, postdocs, and math professors. They contribute because SageMath is directly relevant to their research, and they often contribute state of the art code that implements algorithms they have created or refined as part of their research. Sage is licensed under the GPL, and that license has worked extremely well for us. Academics sometimes even get significant grants from the NSF or the EU to support Sage development.
Companies: I also started the Cython compiler project in 2007, which has had dozens of contributors and is now the defacto standard for writing or wrapping fast code for use by Python. The developers of Cython mostly work at companies (e.g., Google) as a side project in their spare time. (Here's a message today about a new release from a Cython developer, who works at Google.) Cython is licensed under the Apache License.
What RethinkDB Will Become
RethinkDB will no longer be an open source project whose development is sponsored by a single company dedicated to the project. Will it be an academic project, a company-supported project, or dead?A friend of mine at Oxford University surveyed his academic CS colleagues about RethinkDB, and they said they had zero interest in it. Indeed, from an academic research point of view, I agree that there is nothing interesting about RethinkDB. I myself am a college professor, and understand these people! Academic volunteer open source contributors are definitely not going to come to RethinkDB's rescue. The value in RethinkDB is not in the innovative new algorithms or ideas, but in the high quality carefully debugged implementations of standard algorithms (largely the work of bad ass German programmer Daniel Mewes). The RethinkDB devs had to carefully tune each parameter in those algorithms based on extensive automated testing, user feedback, the Jepsen tests, etc.
That leaves companies. Whether or not you like or agree with this, many companies will not touch AGPL licensed code:
"Google open source guru Chris DiBona says that the web giant continues to ban the lightning-rod AGPL open source license within the company because doing so "saves engineering time" and because most AGPL projects are of no use to the company."This is just the way it is -- it's psychology and culture, so deal with it. In contrast, companies very frequently embrace open source code that is licensed under the Apache or BSD licenses, and they keep such projects alive. The extremely popular PostgreSQL database is licensed under an almost-BSD license. MySQL is freely licensed under the GPL, but there are good reasons why people buy a commercial MySQL license (from Oracle) for MySQL. Like RethinkDB, MongoDB is AGPL licensed, but they are happy to sell a different license to companies.
With RethinkDB today, the only option is AGPL. This very strongly discourage use by the only possible group of users and developers that have any chance to keep RethinkDB from death. If this situation is not resolved as soon as possible, I am extremely afraid that it never will be resolved. Ever. If you care about RethinkDB, you should be afraid too. Ignoring the landscape and culture of volunteer open source projects is dangerous.
A Proposal
I don't know who can make the decision to relicense RethinkDB. I don't kow what is going on with investors or who is in control. I am an outsider. Here is a proposal that might provide a way out today:PROPOSAL: Dear RethinkDB, sell me an Apache (or BSD) license to the RethinkDB source code. Make this the last thing your company sells before it shuts down. Just do it.
Hacker News Discussion
Friday, October 7, 2016
RethinkDB, SageMath, Andreessen-Horowitz, Basecamp and Open Source Software
RethinkDB and sustainable business models
Three weeks ago, I spent the evening of Sept 12, 2016 with Daniel Mewes, who is the lead engineer of RethinkDB (an open source database). I was also supposed to meet with the co-founders, Slava and Michael, but they were too busy fundraising and couldn't join us. I pestered Daniel the whole evening about what RethinkDB's business model actually was. Yesterday, on October 6, 2016, RethinkDB shut down.I met with some RethinkDB devs because an investor who runs a fund at the VC firm Andreessen-Horowitz (A16Z) had kindly invited me there to explain my commercialization plans for SageMath, Inc., and RethinkDB is one of the companies that A16Z has invested in. At first, I wasn't going to take the meeting with A16Z, since I have never met with Venture Capitalists before, and do not intend to raise VC. However, some of my advisors convinced me that VC's can be very helpful even if you never intend to take their investment, so I accepted the meeting.
In the first draft of my slides for my presentation to A16Z, I had a slide with the question: "Why do you fund open source companies like RethinkDB and CoreOS, which have no clear (to me) business model? Is it out of some sense of charity to support the open source software ecosystem?" After talking with people at Google and the RethinkDB devs, I removed that slide, since charity is clearly not the answer (I don't know if there is a better answer than "by accident").
I have used RethinkDB intensely for nearly two years, and I might be their biggest user in some sense. My product SageMathCloud, which provides web-based course management, Python, R, Latex, etc., uses RethinkDB for everything. For example, every single time you enter some text in a realtime synchronized document, a RethinkDB table gets an entry inserted in it. I have RethinkDB tables with nearly 100 million records. I gave a talk at a RethinkDB meetup, filed numerous bug reports, and have been described by them as "their most unlucky user". In short, in 2015 I bet big on RethinkDB, just like I bet big on Python back in 2004 when starting SageMath. And when visiting the RethinkDB devs in San Francisco (this year and also last year), I have said to them many times "I have a very strong vested interest in you guys not failing." My company SageMath, Inc. also pays RethinkDB for a support contract.
Sustainable business models were very much on my mind, because of my upcoming meeting at A16Z and the upcoming board meeting for my company. SageMath, Inc.'s business model involves making money from subscriptions to SageMathCloud (which is hosted on Google Cloud Platform); of course, there are tons of details about exactly how our business works, which we've been refining based on customer feedback. Though absolutely all of our software is open source, what we sell is convenience, easy of access and use, and we provide value by hosting hundreds of courses on shared infrastructure, so it is much cheaper and easier for universities to pay us rather than hosting our software themselves (which is also fairly easy). So that's our business model, and I would argue that it is working; at least our MRR is steadily increasing and is more than twice our hosting costs (we are not cash flow positive yet due to developer costs).
So far as I can determine, the business model of RethinkDB was to make money in the following ways: 1. Sell support contracts to companies (I bought one). 2. Sell a closed-source proprietary version of RethinkDB with extra features that were of interest to enterprise (they had a handful of such features, e.g., audit logs for queries). 3. Horizon would become a cloud-hosted competitor to Firebase, with unique advantages that users have the option to migrate from the cloud to their own private data center, and more customizability. This strategy depends on a trend for users to migrate away from the cloud, rather than to it, which some people at RethinkDB thought was a real trend (I disagree).
I don't know of anything else they were seriously trying right now. The closed-source proprietary version of RethinkDB also seemed like a very recent last ditch effort that had only just begun; perhaps it directly contradicted a desire to be a 100% open source company?
With enough users, it's easier to make certain business models work. I suspect RethinkDB does not have a lot of real users. Number of users tends to be roughly linearly related to mailing list traffic, and the RethinkDB mailing list has an order of magnitude less traffic compared to the SageMath mailing lists, and SageMath has around 50,000 users. RethinkDB wasn't even advertised to be production ready until just over a year ago, so even they were telling people not to use it seriously until relatively recently. The adoption cycle for database technology is slow -- people wisely wait for Aphyr's tests, benchmarks comparing with similar technology, etc. I was unusual in that I chose RethinkDB much earlier than most people would, since I love the design of RethinkDB so much. It's the first database I loved, having seen a lot over many decades.
Conclusion: RethinkDB wasn't a real business, and wouldn't become one without year(s) more runway.
I'm also very worried about the future of RethinkDB as an open source project. I don't know if the developers have experience growing an open source community of volunteers; it's incredibly hard and its unclear they are even going to be involved. At a bare minimum, I think they must switch to a very liberal license (Apache instead of AGPL), and make everything (e.g., automated testing code, documentation, etc) open source. It's insanely hard getting any support for open source infrastructure work -- support mostly comes from small government grants (for research software) or contributions from employees at companies (that use the software). Relicensing in a company friendly way is thus critical.
Company Incentives
Companies can be incentived in various ways, including:- to get to the next round of VC funding
- to be a sustainable profitable business by making more money from customers than they spend, or
- to grow to have a very large number of users and somehow pivot to making money later.
For me, SageMath is an open source project I started in 2004, and I'm in it for the long haul. I will make the business I'm building around SageMathCloud succeed, or I will die trying -- therefore I have very, very little tolerance for risk. Failure is not an option, and I am not looking for an exit. For me, the strategy that best matches my values is to incentive my company to build a profitable business, since that is most likely to survive, and also to give us the freedom to maintain our longterm support for open source and pure mathematics software.
Thus for my company, neither optimizing for raising the next round of VC or growing at all costs makes sense. You would be surprised how many people think I'm completely wrong for concluding this.
Andreessen-Horowitz
I spent the evening with RethinkDB developers, which scared the hell out of me regarding their business prospects. They are probably the most open source friendly VC-funded company I know of, and they had given me hope that it is possible to build a successful VC-funded tech startup around open source. I prepared for my meeting at A16Z, and deleted my slide about RethinkDB.I arrived at A16Z, and was greeted by incredibly friendly people. I was a little shocked when I saw their nuclear bomb art in the entry room, then went to a nice little office to wait. The meeting time arrived, and we went over my slides, and I explained my business model, goals, etc. They said there was no place for A16Z to invest directly in what I was planning to do, since I was very explicit that I'm not looking for an exit, and my plan about how big I wanted the company to grow in the next 5 years wasn't sufficiently ambitious. They were also worried about how small the total market cap of Mathematica and Matlab is (only a few hundred million?!). However, they generously and repeatedly offered to introduce me to more potential angel investors.
We argued about the value of outside investment to the company I am trying to build. I had hoped to get some insight or introductions related to their portfolio companies that are of interest to my company (e.g., Udacity, GitHub), but they deflected all such questions. There was also some confusion, since I showed them slides about what I'm doing, but was quite clear that I was not asking for money, which is not what they are used to. In any case, I greatly appreciated the meeting, and it really made me think. They were crystal clear that they believed I was completely wrong to not be trying to do everything possible to raise investor money.
Basecamp
During the first year of SageMath, Inc., I was planning to raise a round of VC, and was doing everything to prepare for that. I then read some of DHH's books about Basecamp, and realized many of those arguments applied to my situation, given my values, and -- after a lot of reflection -- I changed my mind. I think Basecamp itself is mostly closed source, so they may have an advantage in building a business. SageMathCloud (and SageMath) really are 100% open source, and building a completely open source business might be harder. Our open source IP is considered worthless by investors. Witness: RethinkDB just shut down and Stripe hired just the engineers -- all the IP, customers, etc., of RethinkDB was evidently considered worthless by investors.The day after the A16Z meeting, I met with my board, which went well (we discussed a huge range of topics over several hours). Some of the board members also tried hard to convince me that I should raise a lot more investor money.
Will Poole: you're doomed
Two weeks ago I met with Will Poole, who is a friend of a friend, and we talked about my company and plans. I described what I was doing, that everything was open source, that I was incentivizing the company around building a business rather than raising investor money. He listened and asked a lot of follow up questions, making it very clear he understands building a company very, very well.His feedback was discouraging -- I said "So, you're saying that I'm basically doomed." He responded that I wasn't doomed, but might be able to run a small "lifestyle business" at best via my approach, but there was absolutely no way that what I was doing would have any impact or pay for my kids college tuition. If this was feedback from some random person, it might not have been so disturbing, but Will Poole joined Microsoft in 1996, where he went on to run Microsoft's multibillion dollar Windows business. Will Poole is like a retired four-star general that executed a successful campaign to conquer the world; he been around the block a few times. He tried pretty hard to convince me to make as much of SageMathCloud closed source as possible, and to try to convince my users to make content they create in SMC something that I can reuse however I want. I felt pretty shaken and convinced that I needed to close parts of SMC, e.g., the new Kubernetes-based backend that we spent all summer implementing. (Will: if you read this, though our discussion was really disturbing to me, I really appreciate it and respect you.)
My friend, who introduced me to Will Poole, introduced me to some other people and described me as that really frustrating sort of entrepreneur who doesn't want investor money. He then remarked that one of the things he learned in business school, which really surprised him, was that it is good for a company to have a lot of debt. I gave him a funny look, and he added "of course, I've never run a company".
I left that meeting with Will convinced that I would close source parts of SageMathCloud, to make things much more defensible. However, after thinking things through for several days, and talking this over with other people involved in the company, I have chosen not to close anything. This just makes our job harder. Way harder. But I'm not going to make any decisions based purely on fear. I don't care what anybody says, I do not think it is impossible to build an open source business (I think Wordpress is an example), and I do not need to raise VC.
Hacker News Discussion: https://news.ycombinator.com/item?id=12663599
Chinese version: http://www.infoq.com/cn/news/2016/10/Reflection-sustainable-profit-co
Wednesday, October 5, 2016
SageMath: "it's not research"
The University of Washington (UW) mathematics department has funding for grad students to "travel to conferences". What sort of travel funding?
One of my two Ph.D. students at UW asked our Grad program director: "I'll be going to Joint Mathematics Meetings (JMM) to help out at the SageMath booth. Is this a thing I can get funding for?"
ANSWER: Travel funds are primarily meant to support research, so although I appreciate people helping out at the SageMath booth, I think that's not the best use of the department's money.
I think this "it's not research" perspective on the value of mathematical software is unfortunate and shortsighted. Moreover, it's especially surprising as the person who wrote the above answer has contributed substantially to the algebraic topology functionality of Sage itself, so he knows exactly what Sage is.
Sigh. Can some blessed person with an NSF grant out there pay for this grad student's travel expenses to help with the Sage booth? Or do I have to use the handful of 10,ドル 50,ドル etc., donations I've got the last few months for this purpose?
- The department has some money available.
- The UW Graduate school has some money available: They only provide funding for students giving a talk or presenting a poster.
- The UW GPSS has some money available: contact them directly to apply (they only provide funds for "active conference participation", which I think means giving a talk, presenting a poster, or similar)
One of my two Ph.D. students at UW asked our Grad program director: "I'll be going to Joint Mathematics Meetings (JMM) to help out at the SageMath booth. Is this a thing I can get funding for?"
ANSWER: Travel funds are primarily meant to support research, so although I appreciate people helping out at the SageMath booth, I think that's not the best use of the department's money.
I think this "it's not research" perspective on the value of mathematical software is unfortunate and shortsighted. Moreover, it's especially surprising as the person who wrote the above answer has contributed substantially to the algebraic topology functionality of Sage itself, so he knows exactly what Sage is.
Sigh. Can some blessed person with an NSF grant out there pay for this grad student's travel expenses to help with the Sage booth? Or do I have to use the handful of 10,ドル 50,ドル etc., donations I've got the last few months for this purpose?
Tuesday, July 26, 2016
Jupyter: "take the domain name down immediately"
The Jupyter notebook is an open source BSD-licensed browser-based code execution environment, inspired by my early work on the Sage Notebook (which we launched in 2007), which was in turn inspired heavily by Mathematica notebooks and Google docs. Jupyter used to be called IPython.
SageMathCloud is an open source web-based environment for using Sage worksheets, terminals, LaTeX documents, course management, and Jupyter notebooks. I've put much hard work into making it so that multiple people can simultaneously edit Jupyter notebooks in SageMathCloud, and the history of all changes are recorded and browsable via a slider.
Many people have written to me asking for there to be a modified version of SageMathCloud, which is oriented around Jupyter notebooks instead of Sage worksheets. So the default file type is Jupyter notebooks, the default kernel doesn't involve the extra heft of Sage, etc., and the domain name involves Jupyter instead of "sagemath". Some people are disuased from using SageMathCloud for Jupyter notebooks because of the "SageMath" name.
Dozens of web applications (including SageMathCloud) use the word "Jupyter" in various places. However, I was unsure about using "jupyter" in a domain name. I found this github issue and requested clarification 6 weeks ago. We've had some back and forth, but they recently made it clear that it would be at least a month until any decision would be considered, since they are too busy with other things. In the meantime, I rented jupytercloud.com, which has a nice ring to it, as the planet Jupiter has clouds. Yesterday, I made jupytercloud.com point to cloud.sagemath.com to see what it would "feel like" and Tim Clemans started experimenting with customizing the page based on the domain name that the client sees. I did not mention jupytercloud.com publicly anywhere, and there were no links to it.
Today I received this message:
I'm writing this because it's unclear to me what people really want, and I have no idea what to do here.
1. Do you want something built on the same technology as SageMathCloud, but much more focused on Jupyter notebooks?
2. Does the name of the site matter to you?
3. What model should the Jupyter project use for their trademark? Something like Python? like Git?Like Linux? Like Firefox? Like the email program PINE? Something else entirely?
4. Should I be worried about using Jupyter at all anywhere? E.g., in this blog post? As the default notebook for the SageMath project?
I appreciate any feedback.
Hacker News Discussion
UPDATE (Aug 12, 2016): The official decision is that I cannot use the domain jupytercloud.com. They did say I can use jupyter.sagemath.com or sagemath.com/jupyter. Needless to say, I'm disappointed, but I fully respect their (very foolish, IMHO) decision.
SageMathCloud is an open source web-based environment for using Sage worksheets, terminals, LaTeX documents, course management, and Jupyter notebooks. I've put much hard work into making it so that multiple people can simultaneously edit Jupyter notebooks in SageMathCloud, and the history of all changes are recorded and browsable via a slider.
Many people have written to me asking for there to be a modified version of SageMathCloud, which is oriented around Jupyter notebooks instead of Sage worksheets. So the default file type is Jupyter notebooks, the default kernel doesn't involve the extra heft of Sage, etc., and the domain name involves Jupyter instead of "sagemath". Some people are disuased from using SageMathCloud for Jupyter notebooks because of the "SageMath" name.
Dozens of web applications (including SageMathCloud) use the word "Jupyter" in various places. However, I was unsure about using "jupyter" in a domain name. I found this github issue and requested clarification 6 weeks ago. We've had some back and forth, but they recently made it clear that it would be at least a month until any decision would be considered, since they are too busy with other things. In the meantime, I rented jupytercloud.com, which has a nice ring to it, as the planet Jupiter has clouds. Yesterday, I made jupytercloud.com point to cloud.sagemath.com to see what it would "feel like" and Tim Clemans started experimenting with customizing the page based on the domain name that the client sees. I did not mention jupytercloud.com publicly anywhere, and there were no links to it.
Today I received this message:
William,
I'm writing this representing the Jupyter project leadership
and steering council. It has recently come to the Jupyter
Steering Council's attention that the domain jupytercloud.com
points to SageMathCloud. Do you own that domain? If so,
we ask that you take the domain name down immediately, as
it uses the Jupyter name.I'm writing this because it's unclear to me what people really want, and I have no idea what to do here.
1. Do you want something built on the same technology as SageMathCloud, but much more focused on Jupyter notebooks?
2. Does the name of the site matter to you?
3. What model should the Jupyter project use for their trademark? Something like Python? like Git?Like Linux? Like Firefox? Like the email program PINE? Something else entirely?
4. Should I be worried about using Jupyter at all anywhere? E.g., in this blog post? As the default notebook for the SageMath project?
I appreciate any feedback.
Hacker News Discussion
UPDATE (Aug 12, 2016): The official decision is that I cannot use the domain jupytercloud.com. They did say I can use jupyter.sagemath.com or sagemath.com/jupyter. Needless to say, I'm disappointed, but I fully respect their (very foolish, IMHO) decision.
Subscribe to:
Comments (Atom)