ACM Queue - Distributed Development

Program Merge: What's Deep Learning Got to Do with It?: A discussion with Shuvendu Lahiri, Alexey Svyatkovskiy, Christian Bird, Erik Meijer and Terry Coatta

Shuvendu Lahiri, Alexey Svyatkovskiy, Christian Bird, Erik Meijer, Terry Coatta — 2024年9月18日 13:44:42 GMT

I'm Probably Less Deterministic Than I Used to Be: Embracing randomness is necessary in cloud environments.

Pat Helland — 2022年7月24日 14:13:04 GMT

Fail-fast Is Failing... Fast!: Changes in compute environments are placing pressure on tried-and-true distributed-systems solutions.

Pat Helland — 2021年3月25日 11:39:46 GMT

Online Event Processing: Achieving consistency where distributed transactions have failed

Martin Kleppmann, Alastair R. Beresford, Boerge Svingen — 2019年3月24日 16:42:54 GMT

Support for distributed transactions across heterogeneous storage technologies is either nonexistent or suffers from poor operational and performance characteristics. In contrast, OLEP is increasingly used to provide good performance and strong consistency guarantees in such settings. In data systems it is very common for logs to be used as internal implementation details. The OLEP approach is different: it uses event logs, rather than transactions, as the primary application programming model for data management. Traditional databases are still used, but their writes come from a log rather than directly from the application. The use of OLEP is not simply pragmatism on the part of developers, but rather it offers a number of advantages. Consequently, OLEP is expected to be increasingly used to provide strong consistency in large-scale systems that use heterogeneous storage technologies.

Titus: Introducing Containers to the Netflix Cloud: Approaching container adoption in an already cloud-native infrastructure

Andrew Leung, Andrew Spyker, Tim Bozarth — 2017年11月07日 13:57:15 GMT

We believe our approach has enabled Netflix to quickly adopt and benefit from containers. Though the details may be Netflix-specific, the approach of providing low-friction container adoption by integrating with existing infrastructure and working with the right early adopters can be a successful strategy for any organization looking to adopt containers.

Functional at Scale: Applying functional programming principles to distributed computing projects

Marius Eriksen — 2016年9月20日 14:30:47 GMT

Modern server software is demanding to develop and operate: it must be available at all times and in all locations; it must reply within milliseconds to user requests; it must respond quickly to capacity demands; it must process a lot of data and even more traffic; it must adapt quickly to changing product needs; and in many cases it must accommodate a large engineering organization, its many engineers the proverbial cooks in a big, messy kitchen.

The Verification of a Distributed System: A practitioner’s guide to increasing confidence in system correctness

Caitie McCaffrey — 2016年2月01日 21:33:33 GMT

Leslie Lamport, known for his seminal work in distributed systems, famously said, "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." Given this bleak outlook and the large set of possible failures, how do you even begin to verify and validate that the distributed systems you build are doing the right thing?

Testing a Distributed System: Testing a distributed system can be trying even under the best of circumstances.

Philip Maddox — 2015年7月01日 18:21:35 GMT

Distributed systems can be especially difficult to program, for a variety of reasons. They can be difficult to design, difficult to manage, and, above all, difficult to test. Testing a normal system can be trying even under the best of circumstances, and no matter how diligent the tester is, bugs can still get through. Now take all of the standard issues and multiply them by multiple processes written in multiple languages running on multiple boxes that could potentially all be on different operating systems, and there is potential for a real disaster.

Corba: Gone but (Hopefully) Not Forgotten: There is no magic and the lessons of the past apply just as well today.

Terry Coatta — 2008年7月14日 15:03:31 GMT

Corba: Gone But (Hopefully) Not Forgotten

There is no magic and the lessons of the past apply just as well today.

Terry Coatta

Back in the June 2006 issue of Queue, Michi Henning wrote a very good condensed history of CORBA and discussed how some of its technical limitations contributed to its downfall. While those limitations certainly aided CORBA's demise, there is a very widespread notion that the ultimate cause was the ascendance of Web Services, a notion that is compounded with the further belief that Web Services' dominance of the distributed computing landscape is indicative of its technical superiority to the systems that preceded it, such as CORBA and DCOM.

Having worked in distributed systems for a number of years—indeed far enough back in time that building a distributed system meant actually packing bits into UDP packets with lovingly hand-crafted C code—I think this assumption is unwarranted. Worse, it indicates a failure to appreciate the aspects of these previous systems that were well-engineered. This is a symptom of a "silver bullet" mentality that sees Web Services as a radical departure from the past that will finally remove the complexity from designing and building distributed systems.

A Passage to India: Pitfalls that the outsourcing vendor forgot to mention

Mark Kobayashi-Hillary — 2005年2月16日 10:36:56 GMT

A passage to India

Pitfalls that the outsourcing vendor forgot to mention

MARK KOBAYASHI-HILLARY, OFFSHORE ADVISORY SERVICES

Most American IT employees take a dim view of offshore outsourcing. It’s considered unpatriotic and it drains valuable intellectual capital and jobs from the United States to destinations such as India or China. Online discussion forums on sites such as isyourjobgoingoffshore.com are headlined with titles such as “How will you cope?” and “Is your career in danger?” A cover story in BusinessWeek magazine a couple of years ago summed up the angst most people suffer when faced with offshoring: “Is your job next?”

This article, however, is not designed to make the economic case for or against offshore outsourcing. Economists across the world are debating this issue and producing evidence on both sides of the macro-economic fence. What cannot be disputed is that the American IT industry is changing. According to the U.S. Bureau of Labor Statistics, in 1983, computer programmers accounted for 62 percent of all IT jobs in the U.S. with the remainder being computer engineers, analysts, and scientists. By 2002 this ratio had almost reversed, with programmers accounting for just 26 percent of IT jobs.