What is the difference btwn high level and low level libraries?
I understand that keras is a high level library and tensorflow is a low level library but I'm still not familiar enough with these frameworks to understand what that means for high vs low libraries.
Keras is a high level Deep learning(DL) 'API'. Key components of the API are:
Model - to define the Neural network(NN).
Layers - building blocks of the NN model (e.g. Dense, Convolution).
Optimizers - different methods for doing gradient descent to learn weights of NN (e.g. SGD, Adam).
Losses - objective functions that the optimizer should minimize for use cases like classification, regression (e.g. categorical_crossentropy, MSE).
Moreover, it provides reasonable defaults for the APIs e.g. learning rates for Optimizers, which would work for the common use cases. This reduces the cognitive load on the user during the learning phase.
The 'Guiding Principles' section here is very informative:
The mathematical operations involved in running the Neural networks themselves like Convolutions, Matrix Multiplications etc. are delegated to the backend. One of the backends supported by Keras is Tensorflow.
To highlight the differences with a code snippet:
Keras
# Define Neural Network
model = Sequential()
# Add Layers to the Network
model.add(Dense(512, activation='relu', input_shape=(784,)))
....
# Define objective function and optimizer
model.compile(loss='categorical_crossentropy',
optimizer=Adam(),
metrics=['accuracy'])
# Train the model for certain number of epochs by feeding train/validation data
history = model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
Tensorflow
It ain't a code snippet anymore :) since you need to define everything starting from the Variables that would store the weights, the connections between the layers, the training loop, creating batches of data to do the training etc.
You can refer the below links to understand the code complexity with training a MNIST(DL Hello world example) in Keras vs Tensorflow.
https://github.com/keras-team/keras/blob/master/examples/mnist_mlp.py
Considering the benefits that come with Keras, Tensorflow has made tf.keras the high-level API in Tensorflow 2.0.