Understanding people’s complex real-world thinking is a challenge for the behavioral sciences, while computational scientists aim to build systems that can behave intelligently in the real-world. This talk presents a framework for redesigning the everyday websites people interact with to function as: (1) Micro-laboratories for psychological experimentation and data collection, (2) Intelligent adaptive agents that implement machine learning algorithms to dynamically discover how to optimize and personalize people’s learning and reasoning. I present an example of how this framework is used to create “MOOClets” that embed randomized experiments into real-world online educational contexts – like learning to solve math problems. Explanations (and experimental conditions) are crowdsourced from learners, teachers and scientists. Dynamically changing randomized experiments compare the learning benefits of these explanationsin vivowith users, continually adding new conditions as new explanations are contributed. Reinforcement learning algorithms are used for real-time analysis of the effect of explanations on users’ learning, and optimization of the policies for delivering explanations to provide the explanations that are best for different learners. The framework enables a broad range of algorithms for multi-armed bandits to discover how to optimize and personalize users’ behavior, and dynamically adapt technology components to trade off experimentation (exploration) with helping users (exploitation).