[PATCH] staging: lustre: fix sparse warnings related to lock context imbalance
Loïc Pefferkorn
loic at loicp.eu
Sun Dec 7 11:57:03 UTC 2014
On Tue, Dec 02, 2014 at 02:11:33PM -0700, Andreas Dilger wrote:
> On Nov 28, 2014, at 11:50 AM, Greg KH <gregkh at linuxfoundation.org> wrote:
> > On Thu, Nov 27, 2014 at 07:34:10PM +0100, Loïc Pefferkorn wrote:
> >> Hello Greg,
> >>
> >> After some investigation, I think that removing these wrappers is not going to improve the code readability:
> >>
> >> On Wed, Nov 26, 2014 at 12:54:43PM -0800, Greg KH wrote:
> >>> On Wed, Nov 26, 2014 at 05:15:48PM +0100, Loic Pefferkorn wrote:
> >>>> Add __acquires() and __releases() function annotations, to fix sparse warnings related to lock context imbalance.
> >>>>
> >>>> diff --git a/drivers/staging/lustre/lustre/libcfs/hash.c b/drivers/staging/lustre/lustre/libcfs/hash.c
> >>>> index 32da783..7c6e2a3 100644
> >>>> --- a/drivers/staging/lustre/lustre/libcfs/hash.c
> >>>> +++ b/drivers/staging/lustre/lustre/libcfs/hash.c
> >>>> @@ -126,18 +126,21 @@ cfs_hash_nl_unlock(union cfs_hash_lock *lock, int exclusive) {}
> >>>>
> >>>> static inline void
> >>>> cfs_hash_spin_lock(union cfs_hash_lock *lock, int exclusive)
> >>>> + __acquires(&lock->spin)
> >>>> {
> >>>> spin_lock(&lock->spin);
> >>>> }
> >>>>
> >>>> static inline void
> >>>> cfs_hash_spin_unlock(union cfs_hash_lock *lock, int exclusive)
> >>>> + __releases(&lock->spin)
> >>>> {
> >>>> spin_unlock(&lock->spin);
> >>>> }
> >>>
> >>> Ugh, how horrid, please just delete these functions and push down the
> >>> spin_lock/unlock calls down into the places these are called.
> >>
> >> cfs_hash_spin_lock() and cfs_hash_spin_unlock() are referenced by function pointers later in the same file:
> >>
> >> 165 /** no bucket lock, one spinlock to protect everything */
> >> 166 static cfs_hash_lock_ops_t cfs_hash_nbl_lops = {
> >> 167 .hs_lock = cfs_hash_spin_lock,
> >> 168 .hs_unlock = cfs_hash_spin_unlock,
> >> 169 .hs_bkt_lock = cfs_hash_nl_lock,
> >> 170 .hs_bkt_unlock = cfs_hash_nl_unlock,
> >> 171 };
> >> 172
> >> 173 /** spin bucket lock, rehash is enabled */
> >> 174 static cfs_hash_lock_ops_t cfs_hash_bkt_spin_lops = {
> >> 175 .hs_lock = cfs_hash_rw_lock,
> >> 176 .hs_unlock = cfs_hash_rw_unlock,
> >> 177 .hs_bkt_lock = cfs_hash_spin_lock,
> >> 178 .hs_bkt_unlock = cfs_hash_spin_unlock,
> >> 179 };
> >
> > That's even worse than I imagined. Putting sparse markings on these
> > function calls is just papering over nonsense. Please work on
> > unwinding the mess so that you don't need callbacks for locks,
> > that is an abstraction that isn't needed.
>
> Greg,
> in this case these abstractions are not old layering. This is part of
> scalable hashing code that we use all over Lustre, since it needs to
> handle thousands/millions of clients/connections/files/locks/RPCs
> or whatever in a lot of places. Having a single interface to define
> and use these hash functions simplifies the code and avoids bugs,
> like list_* functions do everywhere else in the kernel.
>
> Depending on how the hash table is declared, it may have read/write
> locks or normal spinlocks. The hash table size may be static, or it
> may grow when the buckets get too full (since the numbers of items
> being hashed may differ by many orders of magnitude on different filesystems) so the actual locking functions used depend on how the
> hash table is set up.
>
> I don't think it is possible to unwind the locking of cfs_hash_* code
> since they are accessed via methods depending on the hash.
>
> Cheers, Andreas
>
> >>>> static inline void
> >>>> cfs_hash_rw_lock(union cfs_hash_lock *lock, int exclusive)
> >>>> + __acquires(&lock->rw)
> >>>> {
> >>>> if (!exclusive)
> >>>> read_lock(&lock->rw);
> >>>> @@ -147,6 +150,7 @@ cfs_hash_rw_lock(union cfs_hash_lock *lock, int exclusive)
> >>>>
> >>>> static inline void
> >>>> cfs_hash_rw_unlock(union cfs_hash_lock *lock, int exclusive)
> >>>> + __releases(&lock->rw)
> >>>> {
> >>>> if (!exclusive)
> >>>> read_unlock(&lock->rw);
> >>>
> >>>
> >>> Same for these.
> >>
> >> Likewise for cfs_hash_rw_lock() and cfs_hash_rw_unlock():
> >>
> >> 173 /** spin bucket lock, rehash is enabled */
> >> 174 static cfs_hash_lock_ops_t cfs_hash_bkt_spin_lops = {
> >> 175 .hs_lock = cfs_hash_rw_lock,
> >> 176 .hs_unlock = cfs_hash_rw_unlock,
> >> 177 .hs_bkt_lock = cfs_hash_spin_lock,
> >> 178 .hs_bkt_unlock = cfs_hash_spin_unlock,
> >> 179 };
> >> 180
> >> 181 /** rw bucket lock, rehash is enabled */
> >> 182 static cfs_hash_lock_ops_t cfs_hash_bkt_rw_lops = {
> >> 183 .hs_lock = cfs_hash_rw_lock,
> >> 184 .hs_unlock = cfs_hash_rw_unlock,
> >> 185 .hs_bkt_lock = cfs_hash_rw_lock,
> >> 186 .hs_bkt_unlock = cfs_hash_rw_unlock,
> >> 187 };
> >
> > Same here, ick ick ick.
> >
> >>>> diff --git a/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c b/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c
> >>>> index 2c199c7..1e529fc 100644
> >>>> --- a/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c
> >>>> +++ b/drivers/staging/lustre/lustre/libcfs/libcfs_lock.c
> >>>> @@ -91,6 +91,7 @@ EXPORT_SYMBOL(cfs_percpt_lock_alloc);
> >>>> */
> >>>> void
> >>>> cfs_percpt_lock(struct cfs_percpt_lock *pcl, int index)
> >>>> + __acquires(pcl->pcl_locks[index])
> >>>> {
> >>>> int ncpt = cfs_cpt_number(pcl->pcl_cptab);
> >>>> int i;
> >>>> @@ -125,6 +126,7 @@ EXPORT_SYMBOL(cfs_percpt_lock);
> >>>> /** unlock a CPU partition */
> >>>> void
> >>>> cfs_percpt_unlock(struct cfs_percpt_lock *pcl, int index)
> >>>> + __releases(pcl->pcl_locks[index])
> >>>> {
> >>>> int ncpt = cfs_cpt_number(pcl->pcl_cptab);
> >>>> int i;
> >>>> diff --git a/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c b/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c
> >>>> index 976c61e..257669b 100644
> >>>> --- a/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c
> >>>> +++ b/drivers/staging/lustre/lustre/libcfs/linux/linux-tracefile.c
> >>>> @@ -151,6 +151,7 @@ cfs_trace_buf_type_t cfs_trace_buf_idx_get(void)
> >>>> * for details.
> >>>> */
> >>>> int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
> >>>> + __acquires(&tcd->tc_lock)
> >>>> {
> >>>> __LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
> >>>> if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
> >>>> @@ -165,6 +166,7 @@ int cfs_trace_lock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
> >>>> }
> >>>>
> >>>> void cfs_trace_unlock_tcd(struct cfs_trace_cpu_data *tcd, int walking)
> >>>> + __releases(&tcd->tcd_lock)
> >>>> {
> >>>> __LASSERT(tcd->tcd_type < CFS_TCD_TYPE_MAX);
> >>>> if (tcd->tcd_type == CFS_TCD_TYPE_IRQ)
> >>>> diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c
> >>>> index ce96bd2..8577f97 100644
> >>>> --- a/drivers/staging/lustre/lustre/obdclass/cl_object.c
> >>>> +++ b/drivers/staging/lustre/lustre/obdclass/cl_object.c
> >>>> @@ -193,6 +193,7 @@ static spinlock_t *cl_object_attr_guard(struct cl_object *o)
> >>>> * cl_object_attr_get(), cl_object_attr_set().
> >>>> */
> >>>> void cl_object_attr_lock(struct cl_object *o)
> >>>> + __acquires(cl_object_attr_guard(o))
> >>>> {
> >>>> spin_lock(cl_object_attr_guard(o));
> >>>> }
> >>>> @@ -202,6 +203,7 @@ EXPORT_SYMBOL(cl_object_attr_lock);
> >>>> * Releases data-attributes lock, acquired by cl_object_attr_lock().
> >>>> */
> >>>> void cl_object_attr_unlock(struct cl_object *o)
> >>>> + __releases(cl_object_attr_guard(o))
> >>>> {
> >>>> spin_unlock(cl_object_attr_guard(o));
> >>>> }
> >>>
> >>> Same thing here.
> >>
> >> These ones are easy to replace, but the naming scheme of all functions in cl_object.c is consistent,
> >> from my point of view it ease code reading where they are called, for example in lustre/lustre/osc/osc_request.c:
> >>
> >> before:
> >>
> >> 1827 if (valid != 0) {
> >> 1828 cl_object_attr_lock(obj);
> >> 1829 cl_object_attr_set(env, obj, attr, valid);
> >> 1830 cl_object_attr_unlock(obj);
> >>
> >> after:
> >>
> >> 1827 if (valid != 0) {
> >> 1828 spin_lock(cl_object_attr_guard(obj));
> >> 1829 cl_object_attr_set(env, obj, attr, valid);
> >> 1830 spin_unlock(cl_object_attr_guard(obj));
> >>
> >>
> >> But I'm here for learning, and I would be grateful to have your opinion.
> >
> > Don't hide "implementation of locks" in functions like this, it only
> > causes problems. This code has layers of layers of layers of
> > abstractions due to it wanting to be originally ported to other
> > operating systems and lots of different kernel versions of Linux itself.
> > Unwinding and removing those layers is a good thing to do, don't paper
> > over the nonsense by putting sparse markings on pointless functions.
> >
> > thanks,
> >
> > greg k-h
>
>
> Cheers, Andreas
Hello,
Thanks for your answer. The annotations look like the best thing to do?
--
Cheers,
Loïc
More information about the devel
mailing list