Berkin Ilbeyi has long held a passion for both aspects of computers—software as well as hardware. He researches the intersection of computer software and hardware, with a focus on dynamic programming languages such as Python and JavaScript. These languages, he explains, are frequently used to create new software programs because of their simplicity and ease relative to low-level languages such as C++.

The flipside, however, is that dynamic languages are processed far slower by a computer, making the reasoning behind his dual-pronged approach clear—targeting software for improving the performance of dynamic languages and hardware design methodologies for increasing the performance and ease of computer system design.

Upon completing a double major in electrical and computer engineering and computer science at Lafayette College in Pennsylvania, Ilbeyi joined Christopher Batten’s lab, Electrical and Computer Engineering, at Cornell University, to pursue this work.

“Although my undergraduate study was insightful, I predominantly focused on learning concepts. As a graduate researcher, I have had the opportunity to research new developments and contribute to new findings in the field.”

More Transistors on a Chip versus Software Speed—a Solution

Ilbeyi posits that an example of improving software speed is the case of transistors (semiconductor devices basic to electronic circuits) in processors.

“Since the early 2000s, Moore’s law allowed for more transistors on chip while transistor speeds stagnated, he says.” Hardware designers, however, were unsure about the best way to capitalize on these improvements. Ilbeyi continues, “Designers began adding more cores to processor units. This is useful in cases where problems can be solved in parallel, using multiple cores, but extra cores have little discernible impact in other cases. The question, then, is how can transistors be better utilized?”

A possible solution is to use the extra transistors to observe data patterns within the computer hardware and ascertain which aspects of computation can be performed more efficiently. This information is then fed into the coding framework of the dynamic language—in Ilbeyi’s case, Python—thus allowing for better hardware utilization.

Unlike low-level languages such as C++, Python requires an interpreter program for it to be processed by the computer. It cannot be directly coded into the machine. PyPy, the interpreter Ilbeyi works with, is widely regarded as the fastest one available, especially compared to CPython, its only major alternative. PyPy is also prepackaged with a program known as a Just-In-Time (JIT) compiler, a software which traces frequently used code within Python and optimizes processing time by compiling the repeating instances of code.

“Since the early 2000s, Moore’s law allowed for more transistors on chip while transistor speeds stagnated.”

Ilbeyi explains that the JIT compiler misses several instances of potential compilation. Observing the hardware functions allows Ilbeyi to identify such areas of optimization, feed the information back into the JIT compiler to allow a more speculative version to be re-compiled, and use hardware to ensure the correctness of the re-compiled code.

Techniques for Speeding Up the Hardware

Although improvements in software are usually more ostensible, the hardware design comprises a significant facet of computer systems. “As a computer architecture student, you’re taught to focus on more than simply the functional aspect of programs. Physical elements such as caches and pipelines significantly affect the performance and efficiency of computers as well.”

Hardware designers need detailed simulators to verify correctness and tune their design, but these simulators can be extremely slow. The JIT compiler included with PyPy is a uniquely useful tool for hardware simulation.

“A JIT compiler is a notoriously difficult program to create; it can take years to create an efficient JIT compiler for a program. The PyPy JIT compiler, however, operates on a meta-JIT framework. This means that, whereas a regular JIT compiler focuses on directly optimizing the specific dynamic language, PyPy’s compiler optimizes the interpreter itself, making it easier to repurpose the interpreter framework for languages other than Python.” As long as an interpreter for a language is formulated, it can be optimized by adding PyPy’s JIT compiler to the software.

Amongst other functions, Ilbeyi utilizes PyPy’s meta-JIT framework in the case of machine processors’ executable tasks. “Machine processors frequently execute low-level instructions themselves, in a way not too dissimilar to how Python executes instructions. Both receive a set of instructions for which an interpreter is required.”

Upon ascertaining this similarity, Ilbeyi set about to repurpose PyPy’s JIT compiler framework for machine processing by defining the semantics of processor instructions in the framework. He was able to model physical hardware elements within the same framework, allowing simulations to be both highly accurate and fast.

“The benefits of this technique are immense. For example, I can now run software like ARM programs on a processor with a completely different architecture and that, too, at significantly high simulation speeds and accuracy.”

The Hardware Challenge

Although there have been largely promising results since Ilbeyi began his research, he admits that there remains considerable work to be done. “I’ve run relatively small Python applications nearly thrice as fast as usual speeds, but for more realistically-sized applications, I’m yet to achieve such significant increases.” He says that the pressure to deliver improvements relating to hardware is even greater.

“Hardware design tends to be much more expensive than software design. And unlike software, once a chip is fabricated, it cannot be modified. These factors make the burden for improvement for hardware changes a lot higher. If similar performance increases could be achieved with software-only changes, it is a lot cheaper and preferable. Patience is definitely needed to achieve the targets I’ve set out for the project.”

Following the completion of his PhD, Ilbeyi plans to continue pursuing his passion in California, having accepted a prestigious offer from Google to join their machine-learning project group in February 2019. “Much like the nature of my current research, my project group’s objective will be to formulate an optimized coding structure for Google’s new machine learning hardware system.”