Abstract: There are problems of curse of dimensionality and perceptual aliasing in robot behaviour control under uncertainty. To solve the problem, a framework called episodic memory-driving Markov decision process (EM-MDP) is proposed by introducing neuron stimulation mechanism, in order to achieve environmental experience self-learning and behaviour control under multi-source uncertainty. Firstly, an episodic memory model is built, and an activation and organization mechanism of state neurons is proposed based on cognitive neuroscience. Secondly, self-learning of episodic memory is realized by utilizing adaptive resonance theory (ART) and sparse distributed memory (SDM) through Hebbian rules. A robot behaviour control strategy is established by neuron synaptic potential. Robot can evaluate the past events sequence, predict the current state and plan the desired behaviour. Finally, the experimental results show that the model and control strategy can achieve the objectives of robot behaviour control in universal scenes.