What exactly does word2vec learn?

Berkeley AI Research proposes a quantitative theory of the word2vec learning process: under several mild approximations, its learning problem reduces to unweighted least-squares matrix factorization, the gradient flow dynamics have a closed-form solution, and the final embeddings are equivalent to PCA on the target matrix M*.