Brown CS News

The USENIX Annual Technical Conference (ATC) presented its Best Paper Award this year to a Brown CS team led by doctoral student Evangelos Lamprou and collaborators Lukas Lazarek, a postdoctoral researcher, and faculty member Nikos Vasilakis. Other collaborators include Ethan Williams and Zhuoxuan Zhang of Brown CS, Georgios Kaoukis of the National Technical University of Athens, Michael Greenberg of the Stevens Institute of Technology, and Konstantinos Kallas of UCLA. 

Their paper, “The Koala Benchmarks for the Shell: Characterization and Implications”, published at ATC 2025, was recognized for introducing the first representative collection of Unix shell programs, enabling systematic study of how shell workloads behave across diverse computing domains. This year’s conference was the fiftieth and final edition of ATC, co-located with Operating Systems Design and Implementation (OSDI) 2025 in Boston.

The Unix shell is widely used in data center operations, research computing, and software development, but historically it has lacked a quantitative foundation for evaluating and improving its tools. Evangelos’s Brown-led collaboration between the U.S. and E.U. researchers developed a suite  that includes real-world shell workloads from fields such as AI, DevOps, biotechnology, and the humanities. The work establishes benchmarks that can be used to measure performance, security, and correctness of shell systems, following the principle that progress depends on the ability to measure and compare results.

“Benchmark programs are crucial for evaluating ideas, comparing and contrasting approaches, and fueling academic and industrial research,” Lukas says. “They are especially needed in systems research, where many of the key theses revolve around performance-related arguments and their quantitative evaluation. This need is particularly acute in the context of the shell, where no benchmark suite actually exists.”

ATC has been one of the major venues for software-systems research since its founding. Over five decades, it published influential papers on operating systems, file systems, virtualization, and other areas of computing infrastructure. The 2025 conference marked the end of the ATC series, with the Best Paper Award recognizing research that supports reproducible and quantitative evaluation in systems design.

“While working on POSH, there were no commonly used benchmarks to use and so we instead found our own benchmarks in the wild. It would have been great back then to have a curated list of benchmarks,” says Brown CS faculty member Deepti Raghavan, whose work on POSH (published at ATC '20) was one of the instances that showed how environments like the Shell can result in interesting and potent research. “It's great to see how much this line of work has progressed. I can't wait to see what comes next. I'm sure Koala will enable a lot of interesting ideas and assist future research in this area!”

“We aim for the suite to be useful beyond shell research to the wider systems community,” Evangelos says.

“The talk for this (award-winning!) ATC '25 paper was especially good. Be sure to check it out when the video becomes available!” posted Eric Eide, professor at the University of Utah.

The research originated in Nikos’s research group, whose work focuses on programming languages and systems that make software more efficient and analyzable. Their current projects study how to automate and improve software performance across multiple environments. According to the authors, benchmark programs are essential for evaluating systems ideas and enabling comparison across research efforts, and the new suite is intended to serve both shell and broader systems communities.

For more information, click the link that follows to contact Brown CS Communications Manager Jesse C. Polhemus.