[BUG] staging: rtl8712: possible ABBA deadlocks

[Jia-Ju Bai]

Hello,

My static analysis tool reports several possible ABBA deadlocks in the 
rtl8712 driver in Linux 5.10:

# BUG1:
r8712_free_all_stainfo()
  spin_lock_irqsave(&pstapriv->sta_hash_lock, irqL); --> Line 202 (Lock A)
  r8712_free_stainfo()
    r8712_free_xmitframe_queue()
      spin_lock_irqsave(&(pframequeue->lock), irqL); --> Line 834 (Lock B)

r8711_wx_set_wap()
  spin_lock_irqsave(&queue->lock, irqL); --> Line 1049 (Lock B)
  r8712_set_802_11_infrastructure_mode()
    r8712_free_assoc_resources()
      r8712_get_stainfo()
        spin_lock_irqsave(&pstapriv->sta_hash_lock, irqL); --> Line 228 
(Lock A)

When r8712_free_all_stainfo() and r8711_wx_set_wap() are concurrently 
executed, the deadlocks can occur.

# BUG2:
r8712_joinbss_event_callback()
  spin_lock_irqsave(&pstapriv->sta_hash_lock, irqL2); --> Line 735 (Lock A)
  r8712_free_stainfo()
    r8712_free_xmitframe_queue()
      spin_lock_irqsave(&(pframequeue->lock), irqL); --> Line 834 (Lock B)

r8711_wx_set_wap()
  spin_lock_irqsave(&queue->lock, irqL); --> Line 1049 (Lock B)
  r8712_set_802_11_infrastructure_mode()
    r8712_free_assoc_resources()
      r8712_get_stainfo()
        spin_lock_irqsave(&pstapriv->sta_hash_lock, irqL); --> Line 228 
(Lock A)

When r8712_joinbss_event_callback() and r8711_wx_set_wap() are 
concurrently executed, the deadlocks can occur.

# BUG3:
r8712_stadel_event_callback()
  spin_lock_irqsave(&pstapriv->sta_hash_lock, irqL); --> Line 932 (Lock A)
  r8712_free_stainfo()
    r8712_free_xmitframe_queue()
      spin_lock_irqsave(&(pframequeue->lock), irqL); --> Line 834 (Lock B)

r8711_wx_set_wap()
  spin_lock_irqsave(&queue->lock, irqL); --> Line 1049 (Lock B)
  r8712_set_802_11_infrastructure_mode()
    r8712_free_assoc_resources()
      r8712_get_stainfo()
        spin_lock_irqsave(&pstapriv->sta_hash_lock, irqL); --> Line 228 
(Lock A)

When r8712_stadel_event_callback() and r8711_wx_set_wap() are 
concurrently executed, the deadlocks can occur.


I am not quite sure whether these possible deadlocks are real and how to 
fix them if them are real.
Any feedback would be appreciated, thanks :)

Reported-by: TOTE Robot <oslab@xxxxxxxxxxxxxxx>


Best wishes,
Jia-Ju Bai