Sequence Models and the RNN API (TensorFlow Dev Summit 2017)
Why Stack Overflow recommends it
Stack Overflow users link it when asking or answering questions about TensorFlow's seq2seq decoding helpers, dynamic_decode and RNN input shaping, since the official docs were thin at the time.
Recommended by 2 people across 2 different months, 2017 to 2017.
What it is
A 2017 TensorFlow Dev Summit talk by Eugene Brevdo on building flexible, high-performance sequence-to-sequence models. It covers reading and batching sequence data, the RNN API, fully dynamic unrolling, fused RNN cells, and dynamic decoding.
What people said
Recommends Guillaume Chevalier · 2017-02 · Stack Overflow · on “Predicting next word using the language model tensorflow example”
You seem to want to predict sentences and not only single words. I would recommend you to take a look at Udacity's deep learning class in which there is an assignment where coding a word2vec model itself is done: [LINK] This could help you to figure out how to code a seq2seq model for better sentence prediction with a dynamic decoder. More info here too: [this video]
Recommends Arsene Lupin · 2017-08 · Stack Overflow · on “Is there any difference between TrainingHelper and GreedyEmbeddingHelper in Tensorflow r.1.1(predict result different)?”
Minute 28 in this Tensorflow summit talk [this video] provides some color on the helper classes. As mentioned in Brian's answer - GreedyEmbeddingHelper is meant for prediction time, when ground truth is not available as input. But you can also take a look at ScheduledEmbeddingTrainingHelper, if you want a more nuanced helper at training time
Cites sunside · 2017-02 · Stack Overflow · on “Apart from image and sound recognition, what are real-world use cases for Tensorflow?”
You have to keep in mind that TensorFlow is, as far as the user is concerned, "just" a machine learning API. People may happen to use it for image classification - the 2017 Dev Summit showed medical use cases in skin cancer detection [LINK] and retinal imaging [LINK] - but all the topics of supervised and unsupervised machine learning are candidates for TensorFlow, just like they are for any other ML library; regression of sales by advertisement budget, clustering of users in a social network and recommending books based on previous purchases via collaborative filtering, just to name a few. If you heard about the recent self-driving car projects, think about obtaining steering wheel and brak
Cites Rylan Schaeffer · 2017-06 · Stack Overflow · on “Clarification of TensorFlow's dynamic_decode sample_ids”
tensorflow.contrib.seq2seq 's dynamic_decode returns three values, the first of which is a 2-tuple with named fields 'rnn_output' and sample_id . I'm trying to understand what sample_id is, but I can't find any examples or documentation and the example [this video] at the TensorFlow Developer's Summit doesn't add much information. Can someone explain?
Cites Giuseppe Marra · 2017-06 · Stack Overflow · on “LSTM-RNN : How to shape multivariate Inputs”
I do not know the dataset but I think that you problem is the following: you have a very long sequence and you want to know how to shape this sequence in order to provide this to the network. The 'tf.contrib.rnn.static_rnn' has the following signature: tf.contrib.rnn.static_rnn(cell, inputs, initial_state=None, dtype=None, sequence_length=None, scope=None) where inputs : A length T list of inputs, each a Tensor of shape [batch_size, input_size], or a nested tuple of such elements. So the inputs need to be shaped into lists, where each element of the list is the element of the input sequence at each time step. The length of this list depend on your problem and/or on computational issues. In N
Cites Giuseppe Marra · 2017-08 · Stack Overflow · on “LSTM Autoencoder no progress when script is running on larger dataset”
The second dimension of your input is the number of times the network gets unrolled for computing gradients by the BPTT [LINK] algorithm. The idea is that a recurrent network (like the LSTM) is transformed into a feedforward network by "unrolling" each time step as new layer of the network. When you provide the entire time series together (i.e. 25000 time steps) you are unrolling your network 25000 times, that is you will obtain an unrolled feedforward network with 25000 layers!! So, even though I don't know why you don't get any error, the problem is probably related to an OUT OF MEMORY issue. You are not able to fit 25000 layers' variables into the memory. When you have to deal with long t