🍂 Fall 2022 Update

Improved Parser Error Messages

Haocheng Gao has made significant progress in making Mech's error messages more readable. While the previous update addressed semantic compile-time errors, it was still challenging for new users who frequently encounter syntax errors.

Now, Haocheng has revamped the parser to generate detailed error messages that inform users about the error type, its location in the source file, and possibly offer hints for fixing it. Here's an example of the improved error messages:

Haocheng has tagged various parser combinators to generate relevant and informative messages that are rendered contextually, whether in an editor, console, or browser.

> Mech's parser now produces helpful error messages and makes an effort to recover from common errors. Previously, it only provided a generic error message for invalid inputs. With the new extensions, the parser can display the source code portion containing the error and guide users to identify and fix the issue with annotations and well-crafted error messages.

> This enhancement is part of our ongoing efforts to improve user experience for the upcoming Fall 2022 release. As a Mech learner, I initially struggled with syntax errors due to Mech's unique grammar compared to traditional imperative languages like C and Java, and there was no guidance on correcting my mistakes. I hope our new feature will prevent such difficulties for future learners.

Dynamic Tables

In numerous programs, we can analyze table shapes statically and determine the size of dependent tables at compile time. However, sometimes table shapes depend on variables that might change. Dynamic tables come into play when table sizes need to adapt during runtime. Two common scenarios involving dynamic tables are appending new elements to a table and filtering a table using logical indexing.

Table Flatten Operator (-<)

The table split operator ( >- ) converts a vector into a vector of vectors, which is particularly useful when combined with table interpolation. For instance, splitting a column of numbers can be done as follows:

The result is y = [[1]; [2]; [3]; [4]], and the div will contain:

This functionality has been available for some time. However, the new flatten operator ( -< ) allows us to revert the split vector back to its original form. Here's an example:

The evaluation results in x = [1; 2; 3; 4], which is the original vector.

Update Operators (:+=)

We have introduced a set of update operators to streamline syntax, compiled block code, and memory usage. These operators perform a mathematical operation on a table and update it in place, eliminating the need for an intermediate table. This saves space, time, and lines of code. The update operators are :+= , :-= , :*= , :/= , :^= , and :**= , each corresponding to their respective mathematical operator.

For example:

These new operators have the potential to significantly simplify the syntax. Consider the pose update code from the bouncing balls demo:

With the update operator and table swizzling, this can be condensed into a single line of code:

This example demonstrates how update operators can make code more concise and efficient, while also reducing the need for intermediate tables.

Deprecating and Replacing Title and Subtitle Syntax

We have been concerned about the use of hashtags for header and subheader syntax, as it follows the Markdown convention but could potentially conflict with Mech's global table select operator. To eliminate this potential confusion, we have decided to adopt the alternative Markdown header syntax, which uses an underline of = or - to indicate headers and subheaders, respectively. The new syntax looks like this:

Title
=======

Section Title
---------------
                        

This document's source and the various examples in the Mech examples directory demonstrate the new syntax. By adopting this alternative syntax, we aim to make the Mech language more intuitive and user-friendly, minimizing confusion between different syntax elements.

Boosting Parser Speed (100x Speed Increase)

We identified inefficiencies within the parsing module, specifically the use of String types instead of &str types. By transitioning to &str, we achieved a remarkable 100-fold increase in parser speed. This improvement highlights the elimination of numerous unneeded allocations, leading to a significant performance boost for the Mech language parser!

Smaller Binaries (10x Reduction)

We've achieved a significant reduction in the size of machine binaries, decreasing them by an order of magnitude (from 8MB to 800KB). This improvement was accomplished in two ways:

First, we eliminated redundant code in the dependent repositories. Machines typically depend on core and utilities. The core dependency includes rayon and the entire standard machine, which often go unused in machine implementations. These are now behind feature flags in core. Similarly, utilities has a type definition using a websocket, which depends on tokio. Importing all of tokio for a machine consumed considerable space, so this is now behind a feature flag as well. Removing this unused code accounts for most of the space savings in binaries.

Second, after compiling the binary, we utilized the UPX executable packer to further reduce the machine size (approximately a 50% savings over the regular compiled file).

While we believe there's potential for even greater savings, considering a lot of the code included in these binaries is available in the Mech executable, a long-term task would be to remove this code from the compiled machine and link it to the Mech executable. In the meantime, a 10x reduction is a substantial improvement.

matrix Machine

Introducing the new matrix machine, which offers a range of linear algebra-related functions. By default, matrix/multiply and matrix/transpose are included in the standard machine and made available to programmers through built-in operators. The matrix machine wraps several functions from the Rust nalgebra crate, allowing seamless integration with Mech. This addition enriches Mech's capabilities and provides users with a powerful set of tools for linear algebra operations.

gui Machine

The gui machine incorporates various tables that aid in rendering native interface elements. It primarily utilizes the Rust egui framework, along with custom wrappers that make the elements reactive. Previously, the gui machine was a part of the notebook2 (now simply notebook), but we have since extracted most of its code into a separate machine. As a result, the remaining notebook code is now predominantly implemented using Mech, streamlining the overall organization and functionality.

Haocheng was able to extend the library features enough to be able to draw this angry giraffe:

html machine

Similar to the gui machine, the html machine was previously integrated within the wasm repository. Its primary functions are drawing to canvas and rendering HTML elements. Now, all of the relevant code has been moved into its own separate machine, which has become a dependency of wasm. This change not only improves the code organization but also paves the way for easier maintenance and further development of both the html machine and the wasm repository.

Self-Contained Executables

Mech GUI applications are packaged using the gui crate. This works by having Mech initiate an egui desktop engine and employ one or more Mech cores to update the GUI display in a loop, following a loaded Mech program that defines the App's behavior. However, this design has a limitation as the GUI executable is not self-contained - either a .mec file needs to be separate from the executable or the Mech program has to be included as a string constant within the executable, requiring recompilation each time the Mech program is modified.

To address this issue, the CSE Capstone group has developed a prototype executable builder that enables the creation of a self-contained GUI executable from any .mec file. This solution streamlines the process, allowing for more efficient and user-friendly development of Mech GUI applications.

Notebook resurrection

The previously archived notebook repository has been revived, now hosting the contents of what used to be notebook2. The original notebook has been renamed to wasm-notebook. The primary distinction between these two repositories is their underlying GUI frameworks. The notebook (previously known as notebook2) utilizes the egui Rust GUI framework, while wasm-notebook (previously called notebook) relies on the Rust websys framework. This renaming and reorganization clarify the differences between the two repositories and make it easier for users to choose the appropriate solution based on their preferred GUI framework.

A little confusing, but we can move past that.

Repository reshuffle redux

During the spring, the Mech project repositories were converted into git submodules of the main Mech repo. Recently, several of these git submodules have been relocated to the src directory, and some folder names have been changed. These adjustments aim to streamline the repo's structure, making it cleaner and more user-friendly for developers working on the runtime.

New doc structure

The documentation for Mech is being restructured to improve organization and accessibility. The new layout will be divided into the following categories:

• Getting Started - Welcome, Install, Running, Quickstart, Help

• Reference - Core Language, Writing Programs, Design Documents

• Machines - Standard Machine References

• Guides - Mech-for-X, Tutorials, How-tos

Although the formatting has been updated and the sections outlined, there is still considerable work to be done in these areas. The completion of the documentation is a crucial step before the beta release can be launched.

EKF localization example

The feedback from the IROS submission highlighted the need for a more representative example program to showcase Mech's capabilities. To address this concern, an Extended Kalman Filter (EKF) localization algorithm was implemented in Mech. You can find the example in the repository here. The addition of built-in matrix operators made this possible.

Some interesting aspects of this example include:

• Unicode support is demonstrated through the use of Greek symbols as variable names.

• The Mech implementation of the EKF algorithm is roughly 20 lines of code, which is similar in length to the algorithm expressed in mathematical notation.

• The implementation process was straightforward and closely followed the source algorithm, resulting in a smooth and successful execution.

Moreover, a significant finding from this example is that Mech demonstrates a much faster performance than Matlab. With an EKF localization algorithm implemented in both languages, a direct comparison was made, and Mech significantly outperformed Matlab.

Machine index files

Each machine now includes an index file (written in Mech) that explains the purpose and contents of the machine. It will act as a table of contents for the machine, as well as the first page of documentation. A sample abriged index looks like this:

math
=====

1. Description
---------------

Provides standard mathematical functions.

2. Provided Functions
----------------------

- math/sin(angle)
- math/sin(angle)
  
3. Info
--------

#math/machine = [
  name: "math"   
  version: "v0.1.0"  
  authors: ["Corey Montella", "Haocheng Gao"]   
  machine-url: "https://gitlab.com/mech-lang/machines/math"  
  license: "Apache-2.0"]
                        

You can find the full index here

PLDI ARRAY workshop presentation

On June 13, I had the opportunity to present Mech at the PLDI ARRAY Workshop in San Diego, California, USA. The presentation was an extended version of the 10-minute talk I gave for HYTRADBOI, allowing me to delve deeper into the subject matter and showcase more of Mech's features.

With over twice the amount of time to speak, I was able to provide a more comprehensive overview of Mech, its capabilities, and the problems it aims to solve. Additionally, the longer format allowed me to conduct a live demo, demonstrating some of Mech's unique features and functionality.

Unfortunately, there is no recording of the talk available. However, if you're interested in the content of the presentation, you can refer to the aforementioned HYTRADBOI talk, which covers similar topics and provides a good overview of Mech's capabilities.

Forward Robotics

Forward Robotics is the outreach component of the Mech project, aiming to educate kids about robotics and inspire their interest in STEM fields.

Increased test coverage

While most Mech tests of the core language syntax and semantics are covered in the syntax repository (with almost 200 tests now), we have been gradually expanding the testing of Mech code using the Mech testing framework. Our initial focus is on covering the basic operators to ensure they work with different data types and shapes.

Adding support for more kinds

Our second objective over the summer was to create a matrix of arguments and operators, documenting which work and are tested, which work but are not tested, and which don't work at all. My student DJ Edwards is conducting an independent study with me this semester to help ensure that everything is working and tested before the beta launch.

❤️ Remembering Yuehan Wang

At the end of the summer, tragedy struck with the passing of Yuehan Wang (also known as John). Yuehan worked on Mech with us over the summer, and his contributions were indispensable to the progress we made. Yuehan was a 4th-year undergraduate studying Physics and CS and was very close to completing his degrees before his sudden death. We are all deeply saddened by his loss and extend our heartfelt condolences to his family and friends.