TARDIS Go -> Haxe transpiler

Haxe -> C++ / C# / Java / JavaScript

Project status: a non-working curiosity, development currently on-ice

The advent of Go 1.5, with support for both iOS and Android, together with GopherJS, mean that the cross-platform UI objectives of this project are likely to soon be redundant, as libraries to achieve this can now be written directly in Go. The author has therefore paused development of the Haxe aspect of this project, freezing the runtime at Go 1.4. With the advent of Go 1.6, the code compiles but the tests no-longer pass.

All of the core Go language specification is implemented, including single-threaded goroutines and channels. However the package "reflect", which is mentioned in the core specification, is not yet fully supported.

Goroutines are implemented as co-operatively scheduled co-routines. Other goroutines are automatically scheduled every time there is a channel operation or goroutine creation (or call to a function which uses channels or goroutines through any called function). So loops without channel operations may never give up control. The function runtime.Gosched() provides a convenient way to allow other goroutines to run.

Well over half of the standard packages pass their tests for all targets.

The GOOS for TARDISgo is 'nacl', complete with an in-memory file system. However please note that NaCl provides no access to traditional networking. Go programs written for TARDISgo must use Haxe APIs to access host OS functionality.

A start has been made on the automated integration with Haxe libraries, but this is incomplete and the API unstable, see the haxe/hx directory and gohaxelib repository for the story so far.

The code is developed and tested on OS X 10.10.2, using Go 1.5rc1 and Haxe 3.2.0. The short CI test runs on 64-bit Ubuntu. No other platforms are currently regression tested.

Please note that the Haxe compiler may require in excess of 2Gb of memory to compile a non-trivial program to Java or C#.

To see a handful of live visual examples of code generated by this project in operation, please see the top and right-hand-side of tardisgo.github.io.

Installation and use:

go get -u github.com/tardisgo/tardisgo

If tardisgo is not installing and there is a green "build:passing" icon at the top of this page, please e-mail Elliott!

To translate Go to Haxe, from the directory containing your .go files type the command line:

tardisgo yourfilename.go 

A large number of .hx files will be created in the tardis subdirectory, of which Go.hx contains the entry-point. The use of a file per Haxe class makes second and subsequent compilations using C++ much faster, as only the altered classes are recompiled.

To run your transpiled code you will first need to install Haxe, version 3.2.0.

Then to run the tardis/Go.hx file generated above, for example in JavaScript, type the command lines:

haxe -main tardis.Go -cp tardis -dce full -D uselocalfunctions -js tardis/go.js
node < tardis/go.js

... or whatever Haxe compilation options you want to use. See the tgoall.sh script for simple examples. Note that in this example "-dce full" causes Haxe to do dead code elimination and that "-D uselocalfunctions" is a tardisgo haxe flag to generate JS code that is more likely to be optimized by V8.

The default memory model is fast, but requires more memory than you might expect (an int per byte) and only allows some unsafe pointer usages. If your code uses unsafe pointers to re-use memory as different types (say writing a float64 but reading back a uint64), there is a Haxe compilation flag for "fullunsafe" mode (this is slower, but has a smaller memory footprint and allows most unsafe pointers to be modeled accurately). In JS fullunsafe uses the dataview method of object access, for other targets it simulates memory access. Fullunsafe is little-endian only at present and pointer arithmetic (via uintptr) will panic. A command line example:

tardisgo mycode.go
haxe -main tardis.Go -cp tardis -D fullunsafe -js tardis/go-fu.js
node < tardis/go-fu.js

While on the subject of JS, the closure compiler seems to work, but only using the default "SIMPLE_OPTIMIZATIONS" option. It currently generates a large number of warnings.

The in-memory filesystem used by the nacl target is implemented, it can be pre-loaded with files by using the haxe command line flag "-resource" with the name "local/file/path/a.txt@/nacl/file/path/a.txt" thus (for example in JS):

tardisgo your_code_using_package_os.go
haxe -main tardis.Go -cp tardis -js tardis/go.js -resource testdata/config.xml@/myapp/static/config.xml
node < tardis/go.js

To add more than one file, use multiple -resource flags (the haxe ".hxml" compiler parameter file format can be helpful here). The files are stored as part of the executable code, in a target-specific way. The only resources that will be loaded are those named with a leading "/". A log file of the load process can be found at "/fsinit.log" in the in-memory file-system.

To load a zipped file system (very slow to un-zip, but useful for testing) use go code syscall.UnzipFS("myfs.zip") and include -resource myfs.zip on the haxe command line.

To add Go build tags, use the "-tags 'name1 name2'" tardisgo compilation flag. Note that particular Go build tags are required when compiling for OpenFL using the pre-built Haxe API definitions.

Use the "-debug" tardisgo compilation flag to instrument the code and add automated comments to the Haxe. When you experience a panic in this mode the latest Go source code line information and local variables appears in the stack dump. For the C++ & Neko (--interp) targets, a very simple debugger is also available by using the "-D godebug" Haxe flag, for example to use it in C++ type:

tardisgo -debug myprogram.go
haxe -main tardis.Go -cp tardis -dce full -D godebug -cpp tardis/cpp
./tardis/cpp/Go

To get a list of commands type "?" followed by carriage return, after the 1st break location is printed (there is no prompt character).

To run cross-target command-line tests as quickly as possible, the "-haxe X" flag concurrently runs the Haxe compiler and executes the resulting code as follows:

• "-haxe all" - all supported targets (C++, C#, Java, JavaScript)
• "-haxe bench" - all supported targets (C++, C#, Java, JavaScript) but using benchmark settings
• "-haxe js" - only compiles and runs nodeJS (for automated testing, exits with an error if one occurs)
• "-haxe jsfu" - only compiles (-D fullunsafe) and runs nodeJS (for automated testing, exits with an error if one occurs)
• "-haxe cpp" - only compiles and runs C++ (for automated testing, exits with an error if one occurs)
• "-haxe cs" - only compiles and runs C# (for automated testing, exits with an error if one occurs)
• "-haxe java" - only compiles and runs Java (for automated testing, exits with an error if one occurs)
• "-haxe math" - only runs C++ and JS with the -D fullunsafe haxe flag (using JS dataview)
• "-haxe interp" - only runs the haxe interpreter (for automated testing, exits with an error if one occurs)

Compiler output is suppressed and results appear in the order they complete, with an execution time, for example:

tardisgo -haxe all myprogram.go

When using the -haxe flag with the -test flag, if the file "tgotestfs.zip" exists in the current directory, it will be added as a haxe resource and its contents auto-loaded into the in-memory file system.

If you can't work-out what is going on prior to a panic, you can add the "-trace" tardisgo compilation flag to instrument the code even further, printing out every part of the code visited. But be warned, the output can be huge.

Please note that strings in Go are held as Haxe strings, but encoded as UTF-8 even when strings for that host are encoded as UTF-16. The system should automatically do the translation to/from the correct format at the Go/Haxe boundary, but there are certain to be some occasions when a translation has to be done explicitly (see Force.toHaxeString/Force.fromHaxeString in haxe/haxeruntime.go).

Benchmarks

Tabulating the very simple indicative benchmarking results, looking only at elapsed (rather than cpu) time in seconds, as a multiple of the Go time:

Figures above are the latest results as at 2nd July 2015, including performace figures after running the Google Closure Compiler on the JS and from the parallel project GopherJS (un-minified). The Haxe compilation flag "-D inlinepointers" was used for all targets, the additional flag "-D useloacalfunctions" was used for the JS target.

Execution speed significantly improved after:

• re-writing code generation for non-goroutine functions to reconstruct Haxe "while" and "if" control structures from the SSA form where possible; and
• optimising away local pointers that are only used in a local sub-block and creating temporary local variables to speed execution and reduce code size; however
• since garbage collection currently relies on the target language runtime, the same Haxe code produces very different results.

Expect further gradual speed improvements.

Unsupported Haxe targets: ActionScript, PHP, Python and Neko

The nature of ActionScript/Flash means that it is not possible to run automated tests, although it seems to be a reliable target.

The PHP, Python and Neko targets are not currently reliable enough to permit automated testing.

PHP specific issues:

• to compile for PHP you currently need to add the haxe compilation option "--php-prefix tgo" to avoid name conflicts
• very long PHP class/file names may cause name resolution problems on some platforms

Next steps:

Please go to http://github.com/tardisgo/tardisgo-samples for example Go code modified to work with tardisgo. Including some very simple example code.

For a small technical FAQ, please see the Wiki page.

For public help or discussion please go to the Google Group; or feel free to e-mail Elliott direct to discuss any issues if you prefer.

The documentation is sparse at present, if there is some aspect of the system that you want to know more about, please let Elliott know and he will prioritise that area to add to the wiki.

If you transpile your own code using TARDIS Go, please report the bugs that you find here, so that they can be fixed.

Why do it at all?

The internal objective of the project was for Elliott to learn Go and understand the Go runtime. For both the Go language as a whole, and for the parts of the runtime implemented in TARDISgo, this objective been achieved. However some of the early code that was written would benefit from being refactored.

The external objective of this project was to enable the same Go code to be re-deployed in as many different execution environments as possible, thus saving development time and effort.

However the advent of Go 1.5, with support for both iOS and Android, together with the excellent GopherJS project, mean that the cross-platform UI objectives of this project are now largely redundant.

But it does work. The example code demonstrates both simple command-line examples and multi-platform client-side applications in Go using the APIs available in the Haxe ecosystem:

• The standard Haxe APIs for JavaScript and Flash.
• OpenFL "Open Flash" to target HTML5, Windows, Mac, Linux, iOS, Android, BlackBerry, Firefox OS, Tizen and Flash, using compiled auto-generated C++ code where possible (star users are TiVo and Prezzi).

Target long-term use-cases (if the generated code and runtime environment were developed further to make them more efficient) would be:

• For the Haxe community: provide access to the portable elements of Go's extensive libraries and open-source code base.
• For the Go community: write code in Go and call to-and-from existing Haxe, JavaScript, Java or C# applications (in C++ you would probably just link as normal through CGo).

Future plans:

The project is currently on ice, pending inspiration for developing it further. As at August 2015, it is the only way to target the JVM or CLR from Go.

The Author retains an interest in collaborating in the development of related project ideas, like:

• cross-platform development for Go;
• generating code from Go's SSA form; or
• making Go a target for Haxe.

If you would like to discuss this or any of the above project ideas, that would be wonderful, please contact Elliott.

License:

MIT license, please see the license file.