Roadmap for TensorFlow Extended

[axeltidemann]

The most recent update to the codebase was 10 months ago. What are the future plans for TensorFlow Extended? The TensorFlow ecosystem is still unparalleled when it comes to the machine learning product cycle, from development to production. As TensorFlow is still under active development, I expect TFX to follow suite?

[RyanBalshaw]

+1 on this, we actively make use of the TFX ecosystem internally, but questions around ongoing support, dependency compatibility, and future development are coming up more often.

[RyanBalshaw]

@axeltidemann Do you think its worthwhile escalating? We can @ specific people to find out more.

[axeltidemann]

@RyanBalshaw yes. I don't know who, but please go ahead if you can think of someone.

[RyanBalshaw]

@rcrowe-google, @janasangeetha, @chongkong, @18jeffreyma Hi everyone, could you please assist with this issue, we would love to get an update.

[RyanBalshaw]

@hanneshapke, @casassg Hi guys, I know this might be a stretch, but do you have any information to offer?

[hanneshapke]

I think we need to figure out who is actively using TFX Addons at this point. Google won't push new TFX features at this point. Folks like Robert got assigned to different projects. I think we should convert this project into maintenance mode.

What do you think @RyanBalshaw @casassg ?

[hanneshapke]

Having said that, I have yet to find a framework that offers similar features (e.g., default artifact caching, good visualizations). Any suggestions?

[casassg]

I have since moved on from this tbh (note here I have not been ever at the Google TFX team, i collaborated from my role in another company who used tfx, i am not at that company currently either and my company current does not use it afaik). Also I believe question was on TFX not tfx-addons (which is indeed archived)

[axeltidemann]

@hanneshapke at takealot.com (South Africa's biggest e-commerce site, even with Amazon present) we use TFX extensively for our machine learning products. Like you say, I haven't seen anything that offers similar features like the TensorFlow ecosystem. I would be very surprised if Google does not continue support for it. Not every machine learning problem should be solved with generative AI.

[RyanBalshaw]

@hanneshapke thanks for the response! Sorry for not being clear, the focus of this was on TFX itself, I only noticed yesterday that TFX Addons has been archived. I thought you would be a good addition to this discussion given your history in the TFX platform. I would be sad to see TFX move into maintenance mode, given how useful it is in the work I do with @wihanbooyse.

I recently experimented with KFP SDK v2 and it seems to be a similar representation to TFX without the Tensorflow coupling. It does have artifact caching and it interfaces nicely in the GCP ecosystem. There are certain similar work being done to mimic existing TFX components (e.g., Evaluator) in Google Cloud Pipeline Components, however one key difference is that it is ML-framework agnostic, so I think one might lose the Tensorflow feel/familiarity alongside other features in production.

[RyanBalshaw]

@casassg Thanks for responding! Yeah, similar to Hannes I thought you might just be a useful person to discuss with even though you are not directly tied to Google/TFX. So have you moved on from TFX completely?