Use of TensorFlow

The first article : I am Octopus, the name comes from my Chinese name - octopus; I love programming, algorithms, and open source. All the source code is in my personal github  ; this blog is to record the bits and pieces of my learning, if you are interested in Python, Java, AI, and algorithms, you can follow my news, learn together, and make progress together.

Article directory:

1. TensorFlow's modeling process

2. Core concepts of TensorFlow

3. The hierarchy of TensorFlow

 Fourth, TensorFlow's low-level API

Five, TensorFlow's intermediate API

6. TensorFlow's high-level API


1. TensorFlow 's modeling process

Although TensorFlow is designed to be flexible enough to perform various complex numerical computations.

But generally people use TensorFlow to implement machine learning models, especially neural network models.

In principle, tensors can be used to build computational graphs to define neural networks and train models through automatic differentiation mechanisms.

But for the sake of brevity, it is generally recommended to use TensorFlow's high-level keras interface to implement neural network models.


The general process of implementing a neural network model using TensorFlow includes:

1. Prepare the data

2. Define the model

3. Train the model

4. Evaluate the model

5. Use the model

6. Save the model.

For newbies, the hardest part of it is actually the data preparation process.

The types of data we usually encounter in practice include structured data, image data, text data, time series data.

We will use the titanic survival prediction problem, the cifar2 image classification problem, the imdb movie review classification problem, and the domestic new crown epidemic end time prediction problem as examples to demonstrate the modeling method of these four types of data using tensorflow.


2. Core concepts of TensorFlow

TensorFlow™ is an open source software library for numerical computation using  data flow graphs. Nodes represent mathematical operations in the graph, and edges in the graph represent multidimensional arrays of data that are interconnected between nodes, namely tensors. Its flexible architecture allows you to perform computations on a variety of platforms , such as one or more CPUs (or GPUs) in desktop computers, servers, mobile devices, and more. TensorFlow was originally developed by researchers and engineers in the Google Brain group (part of Google's Machine Intelligence Research Institute) for research in machine learning and deep neural networks , but the generality of the system makes it widely available for other computations as well field .

The main advantages of TensorFlow:

  • Flexibility : support for low-level numerical calculations, C++ custom operators

  • Portability : from server to PC to phone, from CPU to GPU to TPU

  • Distributed computing : distributed parallel computing, operators can be specified to correspond to computing devices

As the saying goes, the building of TensorFlow also has its foundation when the building starts from the ground.

The core concepts at the bottom of Tensorflow are tensors, computational graphs, and automatic differentiation.


3. The hierarchy of TensorFlow

 Introduce 5 different hierarchical structures in TensorFlow: hardware layer, kernel layer, low-level API, middle-level API, and high-level API. Taking the linear regression and DNN binary classification models as examples, the characteristics of the models implemented at different levels are visually compared and displayed.

The hierarchical structure of TensorFlow can be divided into the following five layers from low to high.

The bottom layer is the hardware layer. TensorFlow supports CPU, GPU or TPU to join the computing resource pool.

The second layer is the kernel implemented by C++, and the kernel can be distributed and run across platforms.

The third layer is the operator implemented by Python, which provides low-level API instructions that encapsulate the C++ kernel, mainly including various tensor operation operators, calculation graphs, and automatic differentiation. Such as tf.Variable, tf.constant, tf.function, tf .GradientTape,tf.nn.softmax... If the model is compared to a house, then the third layer API is the [model brick].

The fourth layer is the model component implemented by Python, which encapsulates the low-level API, mainly including various model layers, loss functions, optimizers, data pipelines, feature columns, and so on. Such as tf.keras.layers, tf.keras.losses, tf.keras.metrics, tf.keras.optimizers, tf.data.DataSet, tf.feature_column... If the model is compared to a house, then the fourth layer API It is the [Model Wall].

The fifth layer is the finished model implemented by Python, which is generally a high-level API encapsulated in the OOP way, mainly the class interface of the model provided by tf.keras.models. If the model is compared to a house, then the fifth layer of API is the model itself, that is, the [model house].

                 


 Fourth, TensorFlow's low-level API

TensorFlow's low-level API mainly includes tensor operations, computational graphs, and automatic differentiation.

If the model is compared to a house, then the low-level API is the [model brick].

At the low-level API level, TensorFlow can be used as an enhanced version of numpy.

The methods provided by TensorFlow are more comprehensive than numpy, the operation is faster, and if needed, GPU acceleration can be used.

In the previous chapters, we have had an overall understanding of the low-level API. In this chapter, we will focus on tensor operations and Autograph computation graphs in detail.

The operations of tensors mainly include structural operations of tensors and mathematical operations of tensors.

Tensor structure operations such as: tensor creation, index slicing, dimension transformation, merge and split.

Tensor mathematical operations mainly include: scalar operations, vector operations, and matrix operations. In addition, we will introduce the broadcasting mechanism of tensor operations.

Autograph Computational Graph We will introduce the specification proposal for using Autograph, the mechanism principle of Autograph, Autograph and tf.Module.


Five, TensorFlow's intermediate API

TensorFlow's mid-level API mainly includes:

  • Data pipeline (tf.data)

  • Feature column (tf.feature_column)

  • activation function (tf.nn)

  • Model layers (tf.keras.layers)

  • Loss function (tf.keras.losses)

  • Evaluation function (tf.keras.metrics)

  • Optimizers (tf.keras.optimizers)

  • Callback function (tf.keras.callbacks)

If the model is compared to a house, then the middle-level API is the [model wall].


6. TensorFlow's high-level API

The high-level API of TensorFlow is mainly tensorflow.keras.models.

In this chapter, we mainly introduce the following contents related to tensorflow.keras.models in detail.

  • Model construction (Sequential, functional API, Model subclassing)

  • Model training (built-in fit method, built-in train_on_batch method, custom training loop, single-GPU training model, multi-GPU training model, TPU training model)

  • Model deployment (tensorflow serving deploys the model, uses spark (scala) to call the tensorflow model)

Related: Use of TensorFlow