A tremendously useful explainer about symbolic APIs (Sequential + Functional API) and Model subclassing in TF 2.0, by @random_forests: https://medium.com/tensorflow/what-are-symbolic-and-imperative-apis-in-tensorflow-2-0-dfccecb01021 …
Symbolic APIs are APIs to build graphs of layers. Their strong points are that:
- They match how we think about our networks (NNs are always visualized as graphs of layers in textbooks & papers)
- They run extensive static checks during model construction, like a compiler would
This gives you the guarantee that any model that you can build, will run. The only form of debugging you'd have to do at runtime would be convergence-related. The UX of these APIs is highly intuitive and productive
Meanwhile, the subclassing API has the look and feel of objected-oriented Numpy development. It's ideal if you're doing anything that cannot easily be expressed as a graph of layers, and you feel comfortable with software engineering best practices and large Python projects.
It will involve execution-time debugging, more code, and will expose a greater error surface, but at the same time it will give you greater flexibility to express unconventional architectures.
Importantly, in TF 2.0, both of these styles are available and are fully interoperable. You can mix and match models defined with either style. At the end of the day, everything is a Model!
That way, you are free to pick the most appropriate API for the task at hand.
In general I expect ~90-95% of use cases to be covered by the Functional API. The Model subclassing API targets deep learning researchers specifically (about 5% of use cases).
I think it's great that we don't silo researchers and everyone else into completely separate frameworks. It's all one API, that enables a spectrum of workflows, from really easy (Sequential) to advanced (Functional) to fully flexible and hackable (Model subclassing)
You can follow @fchollet.
Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.
Enjoy Threader? Sign up.
Threader is an independent project created by only two developers. The site gets 500,000+ visits a month and our iOS Twitter client was featured as an App of the Day by Apple. Running this space is expensive and time consuming. If you find Threader useful, please consider supporting us to make it a sustainable project.