TensorFlow was developed for Google’s internal use by Google Brain team, but the system is general enough to be applied to a wide variety of domains. On November 9, 2015, they decided to open source it, and release it under Apache 2.0 open source license. Today we will look into TensorFlow basics and then some Python TensorFlow example programs.
TensorFlow
TensorFlow is a library for dataflow programming. It’s a symbolic math library and is also used for application of machine learning such as neural network.
TensorFlow uses various optimization techniques to make the calculations more efficient. When it comes to computation across several machines and involves huge data sets, TensorFlow offers high scalability. These features put together make TensorFlow a perfect framework for machine learning.
TensorFlow Terminologies
In this post, we will learn more about TensorFlow and work out some examples to see how TensorFlow can be used to visualize, tweak and debug the libraries created with it, but before we get working with TensorFlow we must know what are Tensors and Tensor Processing Units.
Tensors
A tensor is central unit of data in TensorFlow. It consists of primitive values stored in shape of a multidimensional array. The number of dimensions a tensor has is called its rank.
A rank 0 tensor is just a scalar. To keep things simple, we can say that a tensor is rather a fancy name of an array and now we call dimension number as rank. Let’s have a look at what tensors look like:
A rank 1 tensor:
|
1 2 3 |
[1,2,3,4] |
A rank 2 tensor:
|
1 2 3 |
[[1,2,3,4,],[5,6,7,8]] |
Tensor Processing Units (TPUs)
TPU is a programmable AI accelerator designed to provide high throughput of low-precision arithmetic. It is aimed towards utilizing or running models rather than training them. As announced by Google, they have been running TPUs inside their data centres for more than a year and have found them to deliver an order of magnitude better-optimized performance per watt for machine learning.
The second-generation TPUs deliver up to 180 teraflops of performance, and when organized into clusters of 64 TPUs, provide up to 11.5 petaflops.
Getting Started with TensorFlow
We will start by installing Tensorflow.
Installing TensorFlow
We’ll use TensorFlow Python API, it works with Python 2.7 and Python 3.3 and later.
For Linux only: The GPU version requires CUDA 7.0.+ and cuDNN c2+.
Once you have these requirements fulfilled you can begin with installation. For installation we can use either conda or pip, whichever you feel more comfortable with.
The pip installation is standard to install, using pip just run the following command in your terminal (sudo is optional):
|
1 2 3 |
$ pip install tensorflow |
Or you can install using conda, using following command:
|
1 2 3 4 5 |
$ conda create --name TensorflowEnv biopython $ source activate TensorFlowEnv $ conda install tensorflow |
We are installing biopython in first command to make things easier, as it contains some more packages that we might need.
Using TensorFlow
To use TensorFlow anywhere in your script just import it using following import statement.
|
1 2 3 |
import tensorflow |
TensorFlow Simple Expressions
Let’s have a look into simple elements, and how they look like in TensorFlow, before we actually get to work with its elements.
TensorFlow Constants
If you need constants in your training model then you can use constant object. In TensorFlow constants are created using function constant .
The signature of the function is:
|
1 2 3 |
constant(value, dtype=None, shape=None, name="Const", verify_shape=False) |
Here, shape and name are optional parameters. Let’s try to create an object using the signature:
|
1 2 3 |
myConstant = tf.constant(4.5, name="x", dtype=tf.float32) |
TensorFlow Variables
TensorFlow Variables are an important part of any framework as it is impossible to program without variables, they are used to hold and update values.
In TensorFlow variables are of great use when we are training models. As constants, we have to call a constructor to initialize a variable, the initial value can be passed in as an argument. Creating a variable is easy and can be done as following:
|
1 2 3 |
myVariable = tf.Variable(tf.zeros([1]), name="myVariable") |
If we want a variable to be used for calculations only and it should not be trained we can use trainable flag like this:
|
1 2 3 |
k = tf.Variable(tf.add(a, b), trainable=False) |
Variables can easily be added to a computational graph by calling a constructor.
TensorFlow Sessions
A session encapsulates the control and state of the TensorFlow runtime. The session class accepts a graph parameter. If we don’t pass any parameters to a session, it will use the default graph created in the current session.
Let’s have a look at a session in action to know it better:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
# Import tensorflow import tensorflow as tf # Initialize some constants to be used in session x = tf.constant(-2.0, name="x", dtype=tf.float32) a = tf.constant(5.0, name="a", dtype=tf.float32) b = tf.constant(13.0, name="b", dtype=tf.float32) # Declare a Variable y = tf.Variable(tf.add(tf.multiply(a, x), b)) # Assign variable initializer to init init = tf.global_variables_initializer() # Start the session with tf.Session() as session: session.run(init) print(session.run(y)) |
Let’s start the session by running above code in terminal:
The code prints 3.0 which should be the value of y after the computations. Also, please ignore the warning before the output. You might get that as well due to multiple Python versions and other factors.
Defining Computational Graphs
The computation graph is a built-in process in TensorFlow, we need not create an instance of Graph objects explicitly.
In TensorFlow a Graph object can be created with a simple line of code such as:
|
1 2 3 |
c = tf.add(a, b) |
This creates an operation node to sum two numbers.
TensorFlow Placeholder
If we want to inject data into a computation graph we have to use a mechanism named as a placeholder. Placeholders are bound inside some expressions. A typical placeholder looks like:
|
1 2 3 |
placeholder(dtype, shape=None, name=None) |
Placeholders allow us to not to provide the data in advance for operations and computation graphs, and the data can be added in runtime from external sources.
Let’s look at an example. We’ll try to multiply two integers in TensorFlow way, we’ll be using placeholders here:
|
1 2 3 4 5 6 7 8 9 10 |
import tensorflow as tf # Declare placeholders, Notice we didn't pass any values here x = tf.placeholder(tf.float32, name="x") y = tf.placeholder(tf.float32, name="y") z = tf.multiply(x, y, name="z") # Start the session to see the result, and pass values from the feed with tf.Session() as session: print(session.run(z, feed_dict={x: 2.6, y: 2.0})) |
Let’s run the program and see what we get:
Mathematics with TensorFlow
We have already had a brief about tensors, here we’ll see how can we convert a numpy array into a tensor. For the conversion we have to use a built in function convert_to_tensor. This function takes Tensor objects, Numpy arrays, Python lists and Python scalars. Let’s try to convert a 2-d array to tensor.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
import tensorflow as tf import numpy as np # create several numpy arrays tensor_2d = np.array(np.random.rand(4, 4), dtype="float32") tensor_2d_1 = np.array(np.random.rand(4, 4), dtype="float32") tensor_2d_2 = np.array(np.random.rand(4, 4), dtype="float32") # convert to tensor m1 = tf.convert_to_tensor(tensor_2d) m2 = tf.convert_to_tensor(tensor_2d_1) m3 = tf.convert_to_tensor(tensor_2d_2) # perform matrix operation on tensors mat_product = tf.matmul(m1, m2) mat_sum = tf.add(m2, m3) mat_det = tf.matrix_determinant(m3) # run session and see the results with tf.Session() as session: print(session.run(mat_product)) print(session.run(mat_sum)) print(session.run(mat_det)) |
Let’s run the code:
We have tried some matrix operations above with TensorFlow, but how about element-wise operations such as sine, cosine. Well the process remains the same, however if we try a cosine operation on a vector the results so formed would be element-wise. Let’s see this in action with help of an example:
|
1 2 3 4 5 6 7 8 9 10 11 |
import numpy as np import tensorflow as tf # Create a numpy array tensor_1d = np.array([0, 0, 0]) # Convert to tensor tensor = tf.convert_to_tensor(tensor_1d, dtype=tf.float64) # Run and let's find cos of this tensor with tf.Session() as session: print(session.run(tf.cos(tensor))) |
Now, let’s run this:
Machine Learning with TensorFlow
We’ll be looking at a minimal example of Machine Learning and how we can work an algorithm like Linear Regression with TensorFlow.
Linear Regression
Linear regression is a widely used algorithm in the field of machine learning. This algorithm is based on two important concepts of machine learning: Cost Function and Gradient Descent.
So, let’s work this algorithm with TensorFlow:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
import tensorflow as tf import numpy as np # Declare some imporant constants test_data_size = 2000 iterations = 10000 learn_rate = 0.005 # Generate Test Values def generate_test_values(): train_x = [] train_y = [] for _ in range(test_data_size): x1 = np.random.rand() x2 = np.random.rand() x3 = np.random.rand() y_f = 2 * x1 + 3 * x2 + 7 * x3 + 4 train_x.append([x1, x2, x3]) train_y.append(y_f) return np.array(train_x), np.transpose([train_y]) # Create place holders for various values x = tf.placeholder(tf.float32, [None, 3], name="x") W = tf.Variable(tf.zeros([3, 1]), name="W") b = tf.Variable(tf.zeros([1]), name="b") y = tf.placeholder(tf.float32, [None, 1]) model = tf.add(tf.matmul(x, W), b) # Compute cost function cost = tf.reduce_mean(tf.square(y - model)) # Training Model train = tf.train.GradientDescentOptimizer(learn_rate).minimize(cost) train_dataset, train_values = generate_test_values() init = tf.global_variables_initializer() with tf.Session() as session: session.run(init) for _ in range(iterations): session.run(train, feed_dict={ x: train_dataset, y: train_values }) print( "cost = {}".format(session.run(cost, feed_dict={ x: train_dataset, y: train_values }))) print( "W = {}".format(session.run(W))) print( "b = {}".format(session.run(b))) |
Let’s run the program. The output parameters of W and b should be same as those defined in the generate_test_values function:
Python TensorFlow Tutorial Conclusion
In this tutorial we have seen that TensorFlow is a powerful framework and makes it easy to work with several mathematical functions and multidimensional arrays, it also makes it easy to execute the data graphs and scaling.
TensorFlow has grown popular among developers over time. You may need some time to get used to TensorFlow, but once you do so, machine learning on a production scale is less tedious.