Language matters.

Peter Hoddie
  

Peter Hoddie

June 24, 2015

Do computer languages really matter? Any good computer science student can make the argument that most of them are roughly equivalent. But any experienced coder can explain why they consistently chose certain languages for certain tasks. The choice of language matters. It isn’t a matter of theory, it is a matter of practical engineering.

No less an industry legend than Steve Wozniak explained that a defining characteristic of the original Apple Computer was the choice of programming language:

I sniffed the wind and knew that the key to making my computer good (popular) was to include a high-level language and that it had to be BASIC. Engineers programming in FORTRAN were not going to be what would start a home computer revolution.

Screen Shot 2015-07-14 at 4.57.01 PM

Different operating systems have different computer language preferences. Different industries have their preferences.

For a long time now, what we now call the IoT industry, has worked in C, occasionally allowing itself to dabble in C++. Those embedded products typically didn’t connect to a network, or did so in very limited ways. But the Internet is embedded in the name of IoT; embedded products have no choice but to embrace the Internet. To do that, the language needs to change. On the browser, JavaScript is the only language for communications. On servers, JavaScript is a popular choice thanks to node.js, but more broadly scripting languages have dominated for years, from Perl to PHP to Python to Ruby to the more esoteric Erlang.

imgres

Scripts win because they are easy to write, easy to read, and fast to modify. There’s no question that the scripting language with the most momentum in the last decade is JavaScript. And just this month (June 2015) the most significant update to the language has been ratified. Formally known as ECMAScript Sixth Edition or ECMAScript 2015, and informally known as ES6, JavaScript 6th Edition brings the most significant changes to the language since it was introduced in the early days of the Web.

IoT is a revolution. In the next few years, our industry is going to try to connect just about everything to the network. As with the personal computer revolution that Woz propelled forward with BASIC, the IoT revolution needs its high level language to supplant the language of the last generation. And with the advances in JavaScript 6th Edition, that language will be JavaScript. Engineers working in C will be hard pressed to match the speed or creativity of an engineering working in JavaScript.

JavaScript 6th Edition

Work on JavaScript 6th Edition has been going on for years. It involved the efforts of an incredibly skilled and insightful group of engineers collaborating on finding the best way to improve the language. The full specification is huge (nearly 550 pages), and marvelously detailed. Even if you aren’t a computer language fanatic, it is worth a look just to appreciate the precision and attention to detail.

E

The Kinoma team has been monitoring these efforts, knowing that one day we would need to support the language changes. Because we have implemented our own JavaScript engine, XS, adopting a new version of the language means that first we have to update XS. Once that’s done, we can migrate our JavaScript code to take advantage of the new features.

As work on the language stabilized in late 2014, we decided we should begin work on updating XS to support JavaScript 6th Edition. It was inevitable that we would need to support it. Better to do that sooner than later. Because we started early, we had the luxury of time to write and rewrite support for the new features in XS. Because the changes to the language were so significant, we had the opportunity to revisit everything about our JavaScript virtual machine. The result is a considerably faster runtime, that has smaller runtime memory requirements, and excellent compatibility with the JavaScript specification.

What makes JavaScript 6th Edition important for IoT?

By one count, there are about 400 different changes in JavaScript 6th Edition. Each developer will have their own personal favorites. From an IoT product development perspective, these are the features that have made us so enthusiastic:

  • Modules. Modules are now an integral part of the language, rather than a feature added by individual runtimes (like node.js has done so successfully). A single module interface will make sharing and re-using JavaScript code considerably easier. Modules are fundamental to building large-scale systems. When JavaScript was invented, most scripts were just a few lines so long there was no need for modules. But with IoT products that are expected to speak multiple network protocols, connect to dozens of different devices, and connect to an ever changing landscape of web services, modules are a necessity. They are a fundamental tool for minimizing code dependencies.
  • More concise. There have been numerous additions and enhancements to the JavaScript language syntax that reduce the amount of code developers need to write. Common coding patterns that used to take several lines of code are now just a few characters. It is a well documented that the number of bugs in a code base increases with the size of the code base. JavaScript 6th Edition reduces the code.
  • Classes. JavaScript uses a prototype-based inheritance model, which is extremely powerful but unusual. It has often confused programmers coming to JavaScript from languages with more conventional inheritance. With support for classes, JavaScript maintains its prototype-based inheritance but adds a more convention syntax for working with it.
  • Faster. By moving features commonly implemented in scripts into the language, JavaScript 6th Edition has the potential to execute faster.
  • Promises. JavaScript is, by its nature, a single threaded language. To implement asynchronous behaviors, developers have used callbacks. This works, but the code can quickly become a challenging cascade of nested callback functions. Promises are an elegant way of implementing callbacks efficiently.
  • Generators. Generators are a simple mechanism that provides something between cooperative multitasking and streams for JavaScript. They are a little tricky to understand, but we are already seeing how they can considerably simplify the APIs for common operations.
  • Symbols. With the formal introduction of modules, there is a need for modules to attach private data to an object. In the past, code has attached properties with random names to objects to get close to private data. The Symbols object in JavaScript 6th Edition provides true anonymous symbols. Symbols have other intriguing uses as well, which remain for us to explore.

And those are the high points of the language from the Kinoma team’s perspective. There’s more still.

Conformance with the standard

At this point, you may be wondering how many of these features will be available to developers working with KinomaJS? As of this writing (June 23, 2015) most of them. Getting a perfect measure of language conformance for JavaScript 6th Edition is challenging because the language is so new that the formal conformance tests are not yet complete. The most complete language test is the Kangax ECMAScript Compatibility table. Based on those tests, XS6 is at nearly 96% conformance. The next closest engines are at 71% and 66%. You can see our detailed results here.

We’re pleased to have an early head start in implementing JavaScript 6th Edition. We have every confidence that the other engines will catch-up in time. As additional conformance tests become available, we will continue to refine our implementation to be as compatible as practical.

js6-adoption-kinoma_720

Optimized for IoT

Support for the standard is a starting point. There is a great deal of space to innovate around the standard. While most work on JavaScript is concerned with optimizing for the browser or web servers, our focus is on adapting JavaScript for embedded products, to help build better IoT products. Here are some of the areas that the XS6 implementation of JavaScript 6th Edition are optimized for IoT:

  • No JIT. Most JavaScript engines today use a JIT (just in time compiler) to run scripts as fast as possible. JITs do execute scripts spectacularly fast. However, the JIT has to compile the script to native code, which takes time and uses considerable memory. And, due to the dynamic nature of JavaScript, a good JIT is incredibly complex, which means it requires a lot of code. XS6 is pure byte code interpreter. This keeps the runtime small and simple, much more appropriate for memory constrained devices. The IoT developer can code native functions for performance critical functions, so the overall result is fast.
  • Fast runloop. For any interpreter, the runloop is among the most performance critical pieces of code. In XS6, the runloop is optimized in several ways. It is implemented in C, not C++, to minimize runtime overhead, and to keep key state in registers. The byte code instruction dispatch uses labeled C instructions (a so-called threaded dispatch). It avoids recursion on function calls, so a JavaScript function call does not require a C function call, minimizing the native stack and jumps. Further, the XS6 runloop contains inline assembly code for key operations when running on ARM processors. The result is fast, small, portable, and maintainable.
  • Modules. XS6 fully supports modules, and extends it one important way. The module specification defines how to load a module, but makes no provision for unloading a module. On a memory constrained device, that can create a problem as there may well not be enough memory to hold all modules in memory simultaneously that a product uses. XS6 adds a special version of the require function to indicate that a module is eligible for unloaded. The require.weak function loads a module in such a way that when the module is no longer referenced, the garbage collector will automatically unload the module This small addition allows XS6 to significantly reduce the memory requirements of many programs, so they can be run on devices with considerably less memory.
  • Tools. In addition to its JavaScript runtime, XS6 also has a few command line tools. The xsc tool is an offline JavaScript compiler that turns program and module source code into bytecode. xsc has some optimizations built into it, that simplify the byte code deployed to the target device. We expect to expand these optimizations in the future. The xsl tool is the archive utility for XS6. It takes the byte code from multiple programs and modules and links them together in a single archive for deployment. XS6 is able to run code directly from the module, without further loading or linking at runtime. This significantly improves application start-up and module loading time (as much as 4x), and allows byte code to run directly from flash memory (execute in place, e.g. XIP) which reduces RAM usage.
  • Debugging. Eventually, everyone writes code that has a bug. XS6 includes a network debugging protocol that allows source level debugging on most any device XS6 runs on. Because the debugging protocol operates over a TCP connection, remote debugging is always an option. The Kinoma team maintains a GUI XS6 debugger for Mac OS X, Windows, and Linux/GTK.

How little memory?

XS6 contains many optimizations specifically related to reducing memory usage. The total impact of that is considerable. We are able to run nearly the full JavaScript 6th Edition language on a microprocessor (Marvell MW302) that has only 512 KB of RAM. That half megabyte of memory holds the FreeRTOS operating system, networking stack, all dynamic data, and all JavaScript data. On that system, over half the memory is available to scripts. We will provide more detailed memory figures as in the coming weeks. Compare that to a typical JavaScript engine that has been optimized for the browser or a web server, where minimum memory requirements are measured in megabytes, if not tens of megabytes.

There’s no question that operating on a constrained memory device requires discipline. There’s no magic. For example, taking care to split code into modules, and using require.weak to allow unused modules to be garbage collected, makes a big difference. XS6 makes it practical to use JavaScript 6th Edition on embedded devices. As with any engineering task, it requires some knowledge, practice, and care.

The language of IoT

There’s no other programming language today that has as much activity as JavaScript. The work to standardize and evolve the language are unmatched. Work has already begun on the 7th Edition, which looks to improve performance even further. Because of the broad use of the language, specialized efforts, like that of the Kinoma team on XS6, are helping to make JavaScript applicable in more and more places.

This isn’t to say that C is going away. A script will never beat well-optimized C code. For performance critical code and for binding to native platform features, C will continue to dominate. But for more and more application code, networking code, and even hardware interface code, JavaScript is going to be the right choice.

There was a time when JavaScript was the awkward new language. No one was quite sure what to make of it, or whether it would endure. Those days are past. Any developer serious about building IoT products should give JavaScript a hard look. As James Governor of RedMonk observed back in 2010

Try it

If you are find yourself at O’Reilly OSCON in Portland next week, stop by the Kinoma table in the Hardware Showcase. We’ll be showing XS6 powered devices running JavaScript 6th Edition. We’ll have Kinoma Studio, our IDE for KinomaJS development, with XS6 integration, so you can try out the new language syntax. We have also built XS6 Intro, a KinomaJS application that explains many of the JavaScript 6th Edition features with source code examples you can run and modify in real time. xs6-Intro-screenshotWe’ll be making a preview of our JavaScript 6th Edition support available at the end of this month. That will include an update to Kinoma Create and Kinoma Studio so you can begin working in JavaScript 6th Edition. We’ll also post XS6 to our GitHub repository.