I want to summarize Chapters 3 through 5 in Thinking, Fast and Slow by Daniel Kahneman in this post. Chapters 3 and 4 both have connections to teaching. Chapter 5 is absolutely critical to assessing learning.
Chapter 3, The Lazy Controller, emphasizes the idea that it is less stressful in the short term to engage system 1 and not system 2. This may be due to ego deflation. Using system 2 burns short-term glucose stores. It is hard to maintain attention for a long time because these stores get burnt. The chapter also mentions flow. Flow is the state where system 2 focuses on a single problem to the exclusion of most other stimuli. Many view this as a very productive state.
My take away is that simply telling a student to think deeply about a problem might actually increase performance. This agrees with other brain research I’ve read that states that people do not easily transfer an idea from one situation to another unless told to do so. See for instance How People Learn by the National Research Council (0-309-06557-7
Chapter 4, The Associate Machine, tells how what you have done recently has a vast influence on what you do next. Reading words associated with aging actually influenced performance on physical tests, for instance.
At a minimum a teacher could use this priming effect to cause students to think about the ideas in the last class with a quick review.
Chapter 5, Cognitive Ease, tells what may make assessing learning particularly difficult. One theme of the chapter is that the brain does not distinguish the familiar well from the truth. This may be why many political messages seem to be repeated so much. (Tax cuts will spur growth!) Biologically this make sense. It takes energy to remain vigilant using system 2. If an experience has been repeated and it is safe it should arouse less stress and distrust. System 2 can remain at ease.
Unfortunately, when a student is at ease due to repeated experience, clear display, a primed idea or a good mood they commit more errors give less effort to an assessment. On the other hand if a student feels stress about an assignment they are less intuitive and less creative. As a teacher you probably want system 2 engaged so a little stress is probably the side to error upon. You cannot, however, hope for very creative solutions at the same time.
I guess the take away is that an in class timed assessment should include a little stress to get system 2 active. The students need to be vigilant and think about how to approach each problem. To instead assess whether a student can apply the ideas in a more creative way the assessment should probably reduce stress through a clear presentation, connection to other ideas used recently in class and (if possible) a fun presentation that will put students in a good mood.