Update, Jan 2019: Millie Macdonald from our engineering team spoke at ForwardJS, explaining how we ported 100K lines of code to TypeScript. If you missed her talk, we've added her presentation to the end of this post.
Tiny has been developing products in JavaScript for a long while. After trying and rejecting most JavaScript frameworks, we developed our own distinct coding styles in plain, vanilla ES5. Our choices have served us well in both of our JavaScript rich text editors, TinyMCE and Textbox.io. As a result, we avoided a tremendous amount of framework churn and our products have been remarkably stable in the face of major browser changes.
Combining Vanilla JS with our functional programming style caused us to create very defensive code to avoid certain types of common bugs. Unfortunately, with that defensiveness came greatly increased code size, and often performance problems at runtime.
Recent advances in the state of the art suggest it is time for us to up our game. Broad industry agreement on new language features in ES2015 including arrow functions, default parameters, and promises help bring the language closer to mainstream thought around language design. Importantly, ES2015 also finally introduces a standard and built-in module system. It is about time. Every JS developer is tired of the plethora of module systems fit for very particular environments.
Our decision process
According to the State of JavaScript 2017 survey of 20,000 developers, ES2015 migration is well underway in the industry.
Besides migrating our code to ES2015, we decided to seriously investigate alternative JavaScript languages, also known as altJS. We desperately needed two features from functional programming to improve our ability to catch problems during development: static typing (and hence compile-time checking) and pattern matching (to ensure all possible conditions are handled).
Having static typing would allow us to either use or encode two important features in code editors and IDEs. We could find all usages of a function, which is something server-side developers take for granted, and we could get assistance with automated code refactoring. Both of those would significantly speed developer workflow.
We evaluated a number of altJS languages and settled after much debate on a careful combination of ReasonML, PureScript, and TypeScript. Each have their particular strengths. Most importantly, together they give us the combination of functional programming features and ease of migration that our teams demanded.
We had serious and long-lasting discussions before agreeing to three new languages. There are many reasons to be concerned about language diffusion, especially as they relate to training, hiring, and maintenance. Specific areas of discussion included how much of the functional programming feature set we needed and how much we were willing to learn. The learning curve from JavaScript coding, even expert JavaScript coding to “pure” functional programming is long and, frankly, hard. The radically different ways of thinking are coupled with radically different syntactical requirements. These fears were primarily responsible for our separating language usage by task.
We chose ReasonML to rewrite and update the library underlying our PowerPaste plugin for TinyMCE and the ability to copy-and-paste rich text from Microsoft Word into Textbox.io. ReasonML was chosen for this task because the code is difficult, and the temptation to replace our older set of complicated regular expressions with strong types was compelling. We are much more assured that each rule operates just as we think it does. ReasonML has a mature and robust HTML5 parser, which would be an extremely difficult task to undertake internally. Importantly, ReasonML’s syntax is a small leap from vanilla JavaScript compared to more purely functional languages.
We have been using PureScript in some of our internal prototypes for about a year, especially those prototypes produced by the functional programmers on our cloud services team. PureScript implements many functional programming features that are generally not found in other client-side languages for the Web. The use of side-effect tracking would greatly speed code reviews and ease both training and code isolation. Runtime efficiency would improve (a lot) if we used optics. Both of these benefits, and several more, require a language with higher-kinded types. PureScript was our way of bringing these advanced programming concepts into our editors slowly.
However, neither of these languages is a drop-in replacement for our existing JavaScript, yet we identified the need to apply some static typing benefits to our editors. As such, we have just undertaken a large-scale project to convert our editor codebases (spanning thousands of files) to TypeScript.
So why TypeScript?
TypeScript’s killer feature is simply that vanilla JavaScript is legitimate TypeScript. Existing JavaScript libraries can be turned into TypeScript with a bit of wrapping. TinyMCE’s lead engineer wrote a script that does the majority of the work to convert our home-grown module system (based on AMD modules) to ES2015 modules, and with one small <any> annotation our module becomes valid TypeScript:
That exports the module’s functions (in this case, called mouse and keyboard) as the type any. Obviously, we will eventually want to assign individual types for many or even all of our functions, but this approach allows any valid JavaScript code to convert automatically without any overhead.An example diff of a successful migration may be seen in this example from one of our Open Source projects.Most of TypeScript’s type checking features are disabled by assigning all module types as any, but we still get two significant advantages:
- Sometimes a function’s argument length is just wrong in code. Tests, no matter how good, aren’t going to catch every such bug. This entire class of bugs may be caught by the TypeScript compiler even when using type
any. In other cases, we have function with optional arguments. Compiler warnings allow us to identify and clean up such code. - We constrain all libraries to a single API entry point, and only specified that entry point within the TypeScript
includelist – which our conversion script took full advantage of. This was only possible because we had been diligent about creating an “api” folder for the intended API entry points, so after conversion any projects that used non-API modules failed with a compile error.
Just the application of the simplest possible type for our libraries’ functions gave us real and measurable benefit! Many bugs produce parsable JavaScript. Here are some examples that were highlighted even without leveraging the full power of TypeScript.In the first example, missing comment characters around "content" = resulted in the accidental creation of a global variable! The TypeScript compiler highlighted the problem as part of cleaning up optional function parameters.
The second example is a serious logic failure; the isFocused method would fail whenever it was called. The function is located in a rarely-used component that apparently missed out on test coverage.
Perhaps the most extreme example of the TypeScript compiler helping us to uncover a hidden bug was the Case of the Mysterious Double Argument Error™. We had a bug where an inappropriate number of arguments was being passed to a function. Here is the function:
At one point, we made a call to that function like this:
What happened to the keyboard? Oops. A problem with JavaScript is that it lets you get away with these sort of bugs. In this case, the shortcuts variable was assigned to keyboard and the rest cascaded on down the list. The settings variable became undefined.Why didn’t we catch the problem earlier? Embarrassingly, a subsequent bug masked the error, probably as the result of an incomplete refactoring. The keyboard parameter was added to a function call into the module and the inner function where it was used, but there was a chain of function calls between these two methods that did not include it. JavaScript lined up a series of unused parameters along the way purely by accident, thus masking the error. By contrast, the TypeScript compiler helpfully threw a warning about a missing parameter and allowed us to track down and correct the problem.
How will strong typing benefit TinyMCE in the future?
There is no going back. Our initial forays into the use of strong types and type checking on client-side code have already borne fruit. The power of strong typing was critical to the testability of our new copy-and-paste library and has helped find many small, well-hidden bugs in our editors. In the short term, our next steps will be to gradually make more use of TypeScript features and strengthen the type safety of the editor code by:
- Removing the
<any>escape hatches from our automatic conversion - Applying types to functions in our editor libraries and gradually through the core editor as well
- Turning on optional strict compiler features such as strictNullChecks, strictFunctionTypes, and noImplicitReturns to give us better guarantees of type safety
- Building a type definition file for our editor API so TinyMCE is easier to integrate into a TypeScript project
Looking further afield, the success of ReasonML in our copy and paste project has us looking for opportunities to apply the benefits it offers to other parts of our code. There are still plenty of questions to answer before we get there, but we hope to expand our use of it from closed source projects to our open source codebase sometime in 2018.
Sharing with ForwardJS porting 100K lines of code to TypeScript
In early 2019 Millie Macdonald, a software engineer at Tiny, presented a talk at ForwardJS in San Francisco explaining how we ported 100K lines of code to TypeScript. The presentation was based in part on this article by Andy, and her own experiences. If you missed ForwardJS this year, we’re pleased to share Millie’s presentation. Follow Millie on Twitter.
