Merge Gslc into GslCore

[rmcl]

Hey @daz10000,
I'm starting the work of releasing our integration tests and from what I see currently they would need to live in Gslc repo (https://github.com/Amyris/Gslc). I know each of our companies probably have an internal version of that repo that does our own peculiar and proprietary things. That said, as illustrated in this #1 coming up to speed with Gsl as an outside is a real tough lift because the relationship between the repos is not clearly defined and there are no good docs or examples on how to get setup. My perhaps naive proposal is to move the https://github.com/Amyris/Gslc/tree/master/src/Gslc dotnet project in to https://github.com/Amyris/GslCore/tree/master/src. Chris Dolan did some work internally to port that project to dotnetcore from the old mono setup. After that, we could push our integration tests into a test directory in the root of GslCore and they could be run on every change to GslCore.

Do you think this is a good idea? Happy to entertain alternatives if you have better ideas!

[chrismacklin]

Gslc is just a slim wrapper that packages the compiler and the default plugins as an executable. It would be straightforward to write a dedicated integration test runner that operates on the input GSL program and writes all outputs to a scratch directory created for the test run, rather than needing to actually run on the built compiler executable. @daz10000 I'm sure that y'all could whip that up in an hour or so, starting from the existing integration test runner program. That would be a more natural fit for integration into the library-only GslCore.

I think the real work of exporting the integration tests always came down to filtering out the Amryis-specific/proprietary ones, keeping those in-house, and exporting the rest.

[rmcl]

Hi @chrismacklin- Long time no chat, hope all is well!

I think either a dedicated integration test runner or bringing in the old bash script that runs the integration tests works just as well for me! I am willing to do the work to cut out the proprietary stuff from the tests including reworking all the test files to not use the thumper argument. I just need a nice pattern for where to put them and unfortunately I am not sophisticated at F# to cook it up myself.

I hear that GslCore is a library, but one other thing I was thinking it would be interesting to contribute to was more onboarding documentation on how to get started with GSL. If folks from academia or a new company were to come along and want to use GSL, it seems like there is no clear path to get started without the infrastructure that has been built up at Amyris or Demetrix. Do you think it's just a matter of writing a better README in GSLc or are there other ways we can make it easier for developers to get up to speed? I was naively thinking merging Gslc into GslCore as a starter project with some pointers on how to create a proprietary fork of the GSLc dotnet project as users get more advanced would ease the learning curve, but that is just one idea I had on my mind.

Anyway, the second point is somewhat of a secondary concern. Let me know where to put the tests and I will try to make some progress of getting them out into the open :)

My thanks,
Russell

[chrismacklin]

Hi Russell, I'm doing well thank you! And still keeping up with GSLC since it will probably stand as my largest open-source contribution, and it's always nice to keep my feet wet in an actually nice language (I cannot say I recommend go if you like having fun while coding).

Yeah the tests are tricky, as some of the meatiest tests use proprietary compiler plugins to implement operations like crispr. There may also be particular pieces of code such as gene slices used in the tests that could be considered industry secrets. You might want to run them by a veteran strain engineer to make sure that nothing sensitive leaks out.

As far as where to put the tests: probably the best thing would be to land them in a folder in GslCore mirroring the current integration test directory structure, plus the test runner which will of course be broken at first. Then @daz10000 or someone else can restructure the runner as an executable that directly calls the compiler internals.

[chrismacklin]

Also, as far as the onboarding aspect - the idea behind Gslc was that it is basically a starter template for a custom build of the compiler. It is all the code needed to specify your plugins, inject them into the compiler, and package the whole thing up as a CLI. That project seems like a natural place for onboarding documentation. If you need a guide for authoring new plugins, GslCore would be the right spot for that.