Wednesday, January 21, 2009

potential framebuffer deadlock

In the latest kernel (2.6.29-rc2) there's a potential deadlock condition in the frame buffer between fb_ioctl() and fb_mmap().

The cause is that fb_mmap() is called under mm->mmap_sem (A) held, that also acquires fb_info->lock (B); fb_ioctl() takes fb_info->lock (B) and does copy_from/to_user() that might acquire mm->mmap_sem (A) if a page fault occurs.

So we've a classic deadlock condition: a process holds the lock A and attempts to obtain the lock B, but B is already held by a second process that attemps to lock A.

A possible fix is to prevent the deadlock condition, that means "push down" the mutex fb_info->lock into the fb_ioctl() implementation and avoid the occurrence of the page fault with fb_info->lock held (A). But this also requires to define two basic primitives i.e. lock/unlock_fb_info() and use them opportunely inside *all* the framebuffer drivers. For now I've tried to fix at least the main fb_ioctl() function (with the common ioctl ops) that is shared by all the drivers.

No comments: