Neural Nets Remembering

Well yes, all neural nets 'remember'.  We examined that feature in their very early days.    But they don't often remember in the way we would like memory to work.    Very quickly we needed to rework their memory with new infrastructure, based on new context.  Often with new metadata.  So they remember in strict context.   Useful, but you have to be careful about the term.

Following piece is on the topic and interesting

The Neural Network That Remembers
With short-term memory, recurrent neural networks gain some amazing abilities  By Zachary C. Lipton and Charles Elkan ....