The following commit has been merged in the linux branch: commit 6a7b95481d49f73991d3dbf8c1e696a24684ac05 Author: Johannes Weiner hannes@cmpxchg.org Date: Mon Oct 26 16:50:00 2009 -0700

vmscan: order evictable rescue in LRU putback

Isolators putting a page back to the LRU do not hold the page lock, and if the page is mlocked, another thread might munlock it concurrently.

Expecting this, the putback code re-checks the evictability of a page when it just moved it to the unevictable list in order to correct its decision.

The problem, however, is that ordering is not garuanteed between setting PG_lru when moving the page to the list and checking PG_mlocked afterwards:

#0: #1

spin_lock() if (TestClearPageMlocked()) if (PageLRU()) move to evictable list SetPageLRU() spin_unlock() if (!PageMlocked()) move to evictable list

The PageMlocked() check may get reordered before SetPageLRU() in #0, resulting in #0 not moving the still mlocked page, and in #1 failing to isolate and move the page as well. The page is now stranded on the unevictable list.

The race condition is very unlikely. The consequence currently is one page falling off the reclaim grid and eventually getting freed with PG_unevictable set, which triggers a warning in the page allocator.

TestClearPageMlocked() in #1 already provides full memory barrier semantics.

This patch adds an explicit full barrier to force ordering between SetPageLRU() and PageMlocked() so that either one of the competitors rescues the page.

Signed-off-by: Johannes Weiner hannes@cmpxchg.org Reviewed-by: KOSAKI Motohiro kosaki.motohiro@jp.fujitsu.com Cc: Hugh Dickins hugh.dickins@tiscali.co.uk Cc: Mel Gorman mel@csn.ul.ie Cc: Lee Schermerhorn Lee.Schermerhorn@hp.com Cc: Peter Zijlstra a.p.zijlstra@chello.nl Reviewed-by: Rik van Riel riel@redhat.com Signed-off-by: Andrew Morton akpm@linux-foundation.org Signed-off-by: Linus Torvalds torvalds@linux-foundation.org

diff --git a/mm/vmscan.c b/mm/vmscan.c index fbcac3b..777af57 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -544,6 +544,16 @@ redo: */ lru = LRU_UNEVICTABLE; add_page_to_unevictable_list(page); + /* + * When racing with an mlock clearing (page is + * unlocked), make sure that if the other thread does + * not observe our setting of PG_lru and fails + * isolation, we see PG_mlocked cleared below and move + * the page back to the evictable list. + * + * The other side is TestClearPageMlocked(). + */ + smp_mb(); }

/*