[PATCH v2] comedi: integer overflow in do_insnlist_ioctl()

[Xi Wang]

There is a potential integer overflow in do_insnlist_ioctl() if
userspace passes in a large insnlist.n_insns. The call to kmalloc()
would allocate a small buffer, leading to a memory corruption.

Reported-by: Haogang Chen <haogangchen at gmail.com>
Suggested-by: Dan Carpenter <dan.carpenter at oracle.com>
Suggested-by: Ian Abbott <abbotti at mev.co.uk>
Signed-off-by: Xi Wang <xi.wang at gmail.com>
---
 drivers/staging/comedi/comedi_fops.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/comedi/comedi_fops.c b/drivers/staging/comedi/comedi_fops.c
index 21d8c1c..df86a9e 100644
--- a/drivers/staging/comedi/comedi_fops.c
+++ b/drivers/staging/comedi/comedi_fops.c
@@ -670,8 +670,9 @@ static int do_insnlist_ioctl(struct comedi_device *dev,
 		goto error;
 	}
 
-	insns =
-	    kmalloc(sizeof(struct comedi_insn) * insnlist.n_insns, GFP_KERNEL);
+	if (insnlist.n_insns <= ULONG_MAX / sizeof(struct comedi_insn))
+		insns = kmalloc(sizeof(struct comedi_insn) * insnlist.n_insns,
+				GFP_KERNEL);
 	if (!insns) {
 		DPRINTK("kmalloc failed\n");
 		ret = -ENOMEM;
-- 
1.7.5.4