Philly ETE 2016: Securing Software by Construction

Philly ETE 2016: Securing Software by Construction

Editor's Notes

• #2: Hi, I’m Jean, and I’m an academic. This talk is about my research and what I’ve learned trying to get ideas like mine out there into the world. TRANSITION: In case you’re a time traveler, I’m going to start by getting us on the same page.
• #3: Our lives now run on software. Soon we’ll have smart homes, smart cars, and even smart dating. If you haven’t read about the Tinderbox face classifier for automatically swiping right on Tinder, well, I don’t know if you want to. But we first need to "solve" security. Hackers can control our electric skateboards, our rifles, and our cars. Some hacks take years to get fixed and some hacks never get fixed. TRANSTION: The good news is that academic research looks like it’s been making some good progress.
• #4: In academia, people have come up all kinds of solutions to protect our data. There are databases that let you search over encrypted data, mechanisms for replaying programs to find vulnerabilities, tools that can analyze your program for security bugs without even running it, and techniques for building software that has mathematical guarantees of being secure. What’s surprising to me is that the state of the art in industry is firewalls. A firewall is essentially a moat you build around your software to protect it. We’re seeing more and more finer-grained moats, for instance microcontainers, for at the end of the day industry isn’t taking advantage of all the richness that is coming out of academia. The big question I’ve been asking is how can we take advantage of these research ideas in practice. TRANSITION: This talk is about the kinds of ideas people like me are thinking about in academia, why the ideas aren’t flowing out of research, and how we can change this.
• #5: First, I’ll talk about my work as an example of the problems researchers think about. Then I’ll talk what’s preventing security research from being better connected to startups, venture capital and companies—and what I’ve been doing to change this. In trying to connect all the pieces, I realized that software security is an ecosystem problem and that consumers and policy makers are an important part of the solution. Once we connect researchers to everyone else, we can get this nice feedback loop where we’re all working towards this solution of secure software. But the question is how we can connect researchers to everyone else. TRANSITION: I also have some secondary motivations for giving this talk.
• #6: I hear that academics might not be very popular in industry. Apparently we are annoying, have terrible time management, different goals, and not very much money. I’d like to convince you that we’re creative, fun, and great, that we just have different time management and different time scales for our goals. Also, yes, we don’t have money, but if you gave us money that would change the last problem too. TRANSITION: All right, let’s start by talking about what academics like me think about all day long.
• #7: TRANSITION: Before I continue, I should say that I don’t represent all researchers. I’ve been looking at security from a programming languages perspective. This is different than looking at security from a systems or theory perspective!
• #8: I’ve been spending a lot of time thinking about why it’s so hard to protect sensitive data. Here’s a graph showing the hottest data breaches of recent times. We might be saddened, but we’re usually not shocked anymore by these very large numbers of records—hundreds of millions—being leaked by reputable companies like Ebay and Adobe. A big reason these leaks happen is because it’s difficult to reason about the correctness of code and even more difficult to do it at scale. As a result, we have some heuristics to make sure software is more safe than before and most of us have little hope about information security. Obviously, I don’t think things have to be this way because that’s what this talk is about. TRANSITION: Before I describe my proposed solution, let’s look more at why existing approaches aren’t working.
• #9: Well, people can encrypt the data, but often the issue isn’t that people aren’t protecting their data at all, but that they’re showing it under the wrong circumstances. This leaves us with the strategy of finding exploits and developing patches, but this leaves system builders always a step behind. TRANSITION: The goal of my work is to help programmers show data correctly.
• #10: I want to prevent information leaks by factoring privacy out from the rest of the program. I want to allow the programmer to specify high-level policies about how sensitive data can be shown. I want the rest of the program to be agnostic to these policies and I want the system to automatically manage these policies. TRANSITION: Before we talk about how to make this happen, let’s talk a little more about what goes wrong and why.
• #11: The goal of software—or so we’d like—is to keep sensitive information private. The website Airbnb, for turning your apartment into a temporary hotel, has added incentive to do this. As long as you don’t have the phone number of the person you’re contacting, you stay on their site. A lot of the time, when you see a phone number, it will be redacted. TRANSITION: This is the case when you view a message in its entirety, and also when you preview the message in your own email inbox.
• #12: This is the email that was sent to preview the message. Again, Airbnb redacted the number. TRANSITION: The hard thing, however, is that the phone number can show up in so many places!
• #13: This is the screen for previewing messages. Here, the number showed up directly!! (Though I redacted it.) Airbnb tried very hard to prevent phone numbers from leaking, but it missed a spot. TRANSITION: If we think about what goes into preventing leaks, this is not surprising.
• #14: Here are two screen shots from the HotCRP conference management system. I don’t expect you to read the code, but I want you to see how conditional access checks are intertwined with the program. You’ve all probably used this or a similar system to submit and review academic papers. You may be familiar with policies about who can see the titles of papers, the names of authors, and the bodies of reviews. What I’ve highlighted are checks about roles like this. On the right there even dynamically generated SQL queries. In HotCRP, policies are in at least 24 of the 82 files. To implement a policy or to fix a bug, the programmer has to touch many parts of the code. TRANSITION: My solution is to allow programmers to attach policies directly to data.
• #15: For my PhD I worked on a programming model that allows the programmer to attach policies directly to data. The language and runtime manages policies so that programmer does not need to. This approach helps us create programs that are secure by construction, similar to how memory-managed languages yield programs that are memory-safe by construction. TRANSITION: We call this programming model policy-agnostic.
• #16: Here are three screen shots from our model-view-controller policy-agnostic web framework. I don’t expect you to read the code, but I wanted to make the following points. The policies may be centralized. The rest of the program may be agnostic to the policies. The programmer may update policies and other functionality independently. In this programming model, the runtime is responsible for differentiating program behavior. TRANSITION: Now I’m going to tell you how this works.
• #17: The runtime propagates values and policies and solves for values to show based on the policies and the viewer. I picked this snippet of code to illustrate indirect flows. Here we are setting a variable x to zero and incrementing it if the sensitive location value is equal to Chuck E Cheese. Even if the location doesn’t leak directly, the programmer can infer its value by examining x. I’m going to describe how Jeeves prevents indirect flows. We have defined a runtime semantics, proven non-interference, and extended the semantics and guarantees across to the database. TRANSITION: Now I’m going to do a one-slide deep dive.
• #18: Jeeves has a big-step operational semantics that describes, essentially, how to implement a Jeeves interpreter. The semantics models the lambda calculus extended with references and faceted execution. I don’t expect you to understand this slide, but I wanted to demonstrate the level of detail with which we’ve modeled the system. On the top I show evaluation schemas for statements and expressions. Statements include side effects; expressions are pure. On the left-hand side of the down-arrow is the environment sigma and what to evaluate. On the right-hand side is the result. Now I’m going to show the print rule. To the left of the down arrow we have an initial environment sigma and a print statement that is showing expression E_r to output context E_oc. Evaluation produces a policy, output context, and result. This rule shows how we first evaluate the output context and expression to print and then retrieve the relevant labels and policies. Note that because there is always a consistent assignment, we can simply take the transitive closure of labels in the expressions and the relevant policies. We can then evaluate the policies applied to the output context, and then we find an assignment of the labels satisfying the policies and use that to defacet the policy, output context, and result. A high-level take-away is that the policy, output context, and result can all depend on sensitive values and they are all resolved in adherence with the policies. Having a precise model of Jeeves runtime evaluation allows us to characterize the theoretical guarantees, which we’ve also done. TRANSITION: After we figured all this out, we figured we should put our ideas to the test.
• #19: This was around spring 2012 and my advisor was running a student research competition. This is like a conference except there are many, many fewer submissions. So my advisor asked if we could I was pretty sure our research prototype wasn’t ready for the real-world use but I went along with it out of curiosity. I built it and the emails started coming in... At one point, I told my advisor it wasn’t prudent to continue. ... TRANSITION: The good thing was that out of this experience came several years of research, first of all extending the programming model to the database and also optimizing the whole thing.
• #21: The application layer of our web framework runs according to the semantics I described. The programmer is responsible for specifying the policies once, along with the database schemas. The runtime keeps track of the viewer. All values input through the frontend go directly to the database, where they are associated with policies. All values pass back through the Jeeves runtime before display so Jeeves can figure out which value to show. The Jeeves web framework is responsible for keeping track of who is looking. We’ve extended our theoretical model and guarantees to the database, TRANSITION: To give you an idea of how we got to the point where we have a usable web framework, let me give you an idea of timeline.
• #22: TRANSITION: Clearly there are things that academics can do that can help industry, and we know things that nobody else knows yet. The next question is how to push these ideas out into industry.
• #24: This brings us to the next part of our talk, about how to tech transfer these research ideas.
• #25: Some barriers to industry adoption include managers, who need to make economic arguments for change, and programmers, who need to manage legacy code and so can’t go around adopting every new language or tool that comes along. These barriers are more problematic at larger companies, so the startup route to tech transfer is more appealing. And there’s interest in funding startups. Sam Altman, the president of the Ycombinator incubator, recently Tweeted that he would like to fund dozens of security startups in the next couple of years. TRANSITION: So now we just have to turn these ideas into startups.
• #26: But the fact of the matter is, security is no Tindog. Let’s consider Tindog, a hot new Silicon Valley startup that matches dog owners with each other in the style of the Tinder dating application. It has a fun concept and a slick design. It’s so easy to use that your toddler nephew could use it. You don’t have have to change anything about your life to use it. Now let’s think about a startup that goes beyond patching and helps us build more secure systems. The concept is probably going to be highly technical and verifiable only by experts. A good solution will probably require us to rethink a least part of our software infrastructure. TRANSITION: If we want security solutions that do more than help us patch our system.
• #27: The concept is probably highly technical. There are no flashy demos, as the absence of vulnerabilities is harder to show off than the presence of features. These systems often rely on the guarantees this provides, so adoption requires the client to have enough expertise to understand what’s going on—or trust the company to do it right. Finally, the people who come up with these technical problems often do not have the expertise to also build a product. According to Box’s Chief Trust Office Justin Somaini, a major reason security products fail is because they’re made by security people. The stereotype is that "security people" are focused on aspects of the problem other than the user experience. TRANSITION: You may have noticed that this Tweet—and Justin’s t-shirt—has the logo of something called Cybersecurity Factory. This is the accelerator I started with a fellow MIT student to help security startups come into existence.
• #28: This summer, in collaboration with Highland Capital Ventures, we ran the pilot program for Cybersecurity Factory, an 8-week accelerator that gives teams a 20K investment, a network of seasoned entrepreneurs, investors, and potential clients, office space, and legal support. Cybersecurity Factory is also the only accelerator to provide mentorship focused on a technical area. We have recruited a stellar team of industry and academic mentors, including Justin from the previous slide, and also Max Krohn, who founded OKCupid and Keybase, Raj Shah of Palo Alto Networks, and David Ting or Imprivata. Our teams said the mentors were the most useful part of the summer, as the mentors helped them chart paths forward, prevented them from going down dead ends, and provided useful introductions. Our pilot teams are planning to raise funding after this summer. TRANSITION: Something we learned from this accelerator is that doing all this helps companies come into existence and even raise funding, but there’s still a missing piece.
• #29: TRANSITION: If nothing else, it was really educational for our teams to do this program.
• #30: TRANSITION: Here are some more specific things the teams learned.
• #33: TRANSITION: Running our program once has helped confirm our long-term goals.
• #34: Especially for solutions for security by construction that require infrastructure change, we’re going to need to do some work to change how companies get evaluated. TRANSITION: But creating and funding companies doesn’t solve the whole problem.
• #35: The missing piece is how to get companies to care enough about their security to pay for the products these startups are offering. TRANSITION: It’s clear that it’s in everyone’s best interest for software to be more secure.
• #36: According to cybersecurity law professor Jeff Kossef, a report by the Atlantic Council and Zurich Insurance Group estimated that by 2030, an insecure Internet would reduce global economic net benefit by 90ドル trillion, while a completely secure internet would result in a gain of 190ドル trillion. TRANSITION: While we can argue all day about what these estimates really mean, what they are pointing to is a sort of prisoner’s dilemma.
• #37: For those of you who are unfamiliar, the prisoner’s dilemma is a example in game theory where prisoners have varying sentences depending on whether they choose to cooperate or to betray each other. The optimal solution is to cooperate, but the cost of being betrayed is high, so it is theoretically optimal to betray the other person. Similarly, in security, while it’s beneficial for everyone to have more secure software, there is a lack of individual incentive for companies to secure their software. Security requires more employee training, more programmer effort, and also doesn’t currently provide the same competitive advantages that adding new features does. TRANSITION: While there are various policies the government could make to improve the state of things, the important aspect I’ve been thinking about is how to create a culture around caring more.
• #38: Right now, companies are getting the message that they can get away with not securing their software. First, consumers need to show that they are serious about security and privacy. Despite its egregious privacy violations, the ephemeral photo messaging application Snapchat is valued at 16 billion and 100 million. The price of their violations was a few weeks of bad press. The Federal Trade Commission seemed to be the only people who really cared. I was the only person I knew who uninstalled Snapchat not just because I passed age 13, but to boycott their disregard for privacy. Second, policy enforcers need to show they are serious about catching enforcement. One of our Cybersecurity Factory teams spent some time interviewing dentists about their HIPAA compliance and discovered that it is common practice to send X-rays and other records via email, in violation of the privacy standards. Part of this was simply because there was confusion about what compliance means, but people also cited lack of audit risk as a reason not to take privacy standards more seriously. They found that hospital CISOs face similarly low audit risk. For a good example of policy enforcement, we can look to the FTC, which automatically detects when questionable trade mergers. We can imagine understanding better the sources of leaks and building tools to better enforce government privacy policies. TRANSITION: Looking forward, you can take home and apply the following recipe for securing software.
• #39: Some things legislation can do: Put expiration date on data. Allow people to opt out of data collection. TRANSITION: By now, I hope I’ve convinced you that we should want to secure our software by construction and that we’ll need to solve many problems to do it.
• #40: First, I’ll talk about my work as an example of the problems researchers think about. I’ll talk what’s preventing security research from being better connected to startups, venture capital and companies. Finally, I’ll talk about why policy makers and consumers are part of the solution. The big question is how we can connect researchers to everyone else. TRANSITION: Let’s start by talking about what researchers already know.
• #41: It’s really exciting that software has become such a dominant force in our lives. We’re going to have to do some work to create the right ecosystem so we can secure our software by construction, but it’s going to be worth it. I look forward to seeing what we can do!