XVME 6300 with TSI148 bridge on 64 bit Debian (Linux 3.2.57) vme_user issue

Maurice Moss eightplusclub at gmail.com
Tue Nov 18 16:53:30 UTC 2014


Hi Martyn,

Any update on this from your side?

Cheers!

On Mon, Nov 10, 2014 at 10:22 AM, Maurice Moss <eightplusclub at gmail.com> wrote:
> Hi Martyn,
>
> Thanks for your reply. I have pasted the code below. It is very much
> similar to your test code from the forum.
>
> Thanks!
>
> #define _XOPEN_SOURCE 500
> #define u32 unsigned int
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/ioctl.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include "vme_user.h"
> int main(int argc, char *argv[])
> {
>         int fd;
>         int i;
>         int retval;
>         int readSize = 2;
>         unsigned char data[readSize];
>
>         struct vme_master master;
>
>         printf("Simple VME User Module Test\n");
>
>         fd = open("/dev/bus/vme/m0", O_RDONLY);
>         if (fd == -1) {
>                 perror("ERROR: Opening window device file");
>                 return 1;
>         }
>
>         printf ("opened the file\n");
>         master.enable = 1;
>         //        master.vme_addr = 0x114000;
>         master.vme_addr = 0x114000;
>         master.size = 0x10000;
>         master.aspace = 0x2; // VME_A24
>         master.cycle = 0x2000 | 0x8000;// user/data access
>         master.dwidth = 0x2; // 16 bit word access
>
>         retval = ioctl(fd, VME_SET_MASTER, &master);
>         if (retval != 0) {
>                 printf("retval=%d\n", retval);
>                 perror("ERROR: Failed to configure window");
>                 return 1;
>         }
>
>         printf ("set the master\n");
>         /*
>          * Reading first 512 bytes
>          */
>         for (i=0; i<readSize; i++) {
>                 data[i] = 0;
>         }
>
>         retval = pread(fd, data, readSize, 0x03c);
>         if (retval < readSize) {
>                 printf("WARNING: Only read %d bytes", retval);
>         }
>
>         for(i=0; i<retval; i++) {
>                 if (i % 8 == 0) {
>                         printf("\n""%4.4x: ", i);
>                 }
>                 printf("%2.2x ", data[i]);
>         }
>         printf("\n");
>
>         close(fd);
>
>         return 0;
> }
>
> On Mon, Nov 10, 2014 at 4:07 AM, Welch, Martyn (GE Intelligent
> Platforms) <martyn.welch at ge.com> wrote:
>> Hi Maurice,
>>
>> Would you be able to point to a complete piece of test code that exhibits this failure?
>>
>> Martyn Welch
>> Lead Software Engineer
>> GE Intelligent Platforms
>> Embedded Systems
>>
>> T +44 (0) 1327 322748
>> F +44 (0) 1327 322817
>> E martyn.welch at ge.com
>>
>> Tove Valley Business Park
>> Towcester, Northants, NN12 6PF, United Kingdom
>> GE Intelligent Platforms Ltd
>>
>> GE imagination at work
>>
>> GE Intelligent Platforms Ltd, registered in England and Wales (3828642) at 100 Barbirolli Square, Manchester, M2 3AB, VAT GB 927 5591 89
>>
>>
>>> -----Original Message-----
>>> From: Maurice Moss [mailto:eightplusclub at gmail.com]
>>> Sent: 08 November 2014 00:33
>>> To: Welch, Martyn (GE Intelligent Platforms)
>>> Cc: Manohar Vanga; dan.carpenter at oracle.com; driverdev-
>>> devel at linuxdriverproject.org
>>> Subject: Re: XVME 6300 with TSI148 bridge on 64 bit Debian (Linux 3.2.57)
>>> vme_user issue
>>>
>>> Hi Manohar/Dan,
>>>
>>> Any idea regarding this?
>>>
>>> Cheers,
>>> Maurice
>>>
>>> On Mon, Nov 3, 2014 at 5:25 PM, Maurice Moss <eightplusclub at gmail.com>
>>> wrote:
>>> > Hi Martyn,
>>> >
>>> > Thanks for your help from previous emails.  I managed to talk to my
>>> > board using a VME-USB board. Now I am back to working with an SBC, and
>>> > I have a different setup this time around, let me describe it:
>>> >
>>> > 1. SBC in slot 0 of a VME64 chassis (with 2 slots), and the bottom one
>>> > being a slot for an SBC.  The SBC is has a Universe-II and when I load
>>> > the kernel module manually, everything seems fine, and I see this in
>>> > dmesg:
>>> > [   76.192738] vme_ca91cx42 0000:02:04.0: enabling device (0140 -> 0143)
>>> > [   76.192893] vme_ca91cx42 0000:02:04.0: Board is the VME system
>>> controller
>>> > [   76.192902] vme_ca91cx42 0000:02:04.0: Slot ID is 0
>>> > [   76.192907] vme_ca91cx42 0000:02:04.0: CR/CSR Offset: 0
>>> > [   76.192911] vme_ca91cx42 0000:02:04.0: Slot number is unset, not
>>> > configuring CR/CSR space
>>> > [   76.195956] vme_ca91cx42 0000:02:04.0: CR/CSR configuration failed.
>>> >
>>> > I don't intend to use CR/CSR feature.  The linux kernel I am running
>>> > is 3.13, the board is essentially this:
>>> > http://www.onestopsystems.com/documents/OSS-PCIe-KIT-6400.pdf
>>> >
>>> > 2. Now I would like to talk to a passive Slave board in slot 1 (I am
>>> > not sure about this numbering, basically the board in the other slot).
>>> > This slave board essentially talks only A24 and D16 in
>>> > user/super/data.  It's address space internally begins at 0x114000. In
>>> > my test code, I essentially have the following:
>>> >
>>> >         master.enable = 1;
>>> >         //        master.vme_addr = 0x114000;
>>> >         master.vme_addr = 0x114000;
>>> >         master.size = 0x10000;
>>> >         master.aspace = 0x2; // VME_A24
>>> >         master.cycle = 0x2000 | 0x8000;// user/data access
>>> >         master.dwidth = 0x2; // 16 bit word access
>>> >         retval = ioctl(fd, VME_SET_MASTER, &master);
>>> >
>>> > The call doesn't fail, and when I make a pread, all I get are 0xff s
>>> > on every byte.
>>> > I feel like I just can't seem to get the vme_addr to point in the
>>> > right direction. I know it's not the slave board, as I have verified
>>> > that it works with the VME-to-USB.
>>> >
>>> > In my mind, I have to set the SBC as a VME master and make a read at
>>> > A24 address.  However, in vme_user.c I notice that the master resource
>>> > is allocated as A32.  Which is why I just can't seem to get the whole
>>> > addressing schema right!
>>> >
>>> > Here is my lspci -v
>>> >
>>> > 02:04.0 Bridge: Tundra Semiconductor Corp. CA91C042 [Universe] (rev 02)
>>> >         Flags: medium devsel, IRQ 16
>>> >         Memory at f7d00000 (32-bit, non-prefetchable) [size=4K]
>>> >         I/O ports at e000 [size=4K]
>>> >         Kernel driver in use: vme_ca91cx42
>>> > 08:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8113 (rev 01)
>>> > (prog-if 01 [Subtractive decode])
>>> >         Flags: bus master, fast devsel, latency 0
>>> >         Bus: primary=08, secondary=09, subordinate=09, sec-latency=64
>>> >         Memory behind bridge: f7000000-f78fffff
>>> >         Prefetchable memory behind bridge: 00000000f6000000-
>>> 00000000f6ffffff
>>> >         Capabilities: <access denied>
>>> >
>>> > Any help is as usual thoroughly appreciated. And in addition thanks
>>> > for all your help already!
>>> >
>>> > Cheers,
>>> > Maurice
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Thu, Jul 24, 2014 at 1:45 AM, Martyn Welch <martyn.welch at ge.com>
>>> wrote:
>>> >> On 23/07/14 03:09, Maurice Moss wrote:
>>> >>>
>>> >>> Hi Martyn,
>>> >>>
>>> >>> Thanks for your patience with me.  I have a couple of questions for you:
>>> >>>
>>> >>> 0. I put the SBC with the right settings for Geographical addressing.
>>> >>> I did 2 tests by setting the board in each of the 2 slots available
>>> >>> on my rack and the geo address was detected as 0 in both the cases.
>>> >>> This means my backplane isn't working or that my SBC isn't talking
>>> >>> to the backplane.
>>> >>
>>> >>
>>> >> What settings did you apply to "set" geographical addressing? Is this
>>> >> the drivers or something board specific?
>>> >>
>>> >>> 1. Is there a way I can test whether the PCI bridge is working?
>>> >>
>>> >>
>>> >> I assume you mean whether the PCI bridges are passing the PCI address
>>> >> ranges used by the VME windows through to the device?
>>> >>
>>> >> It think "lspci -v" will show you what ranges the bridges have, you
>>> >> will probably need to stick some debug into vme_tsi148.c to get the
>>> >> pci_base address as allocated in tsi148_master_set().
>>> >>
>>> >> This can be very board dependant, so I'm afraid I can't help much here.
>>> >>
>>> >>> 2. I don't understand what should be the exact vme base address of
>>> >>> my slave board.  I am now using VDIS8004 set in slot 2,
>>> >>> (http://www.ifh.de/~wischnew/amanda/daq/ces_8004_v10_.pdf) set
>>> to
>>> >>> VME short A16 (The static rotatory switches set to 2 and 2).  Based
>>> >>> on this my address would be 0x2200?  Any clarification or pointing
>>> >>> me in the right direction would be sincerely appreciated :-/
>>> >>
>>> >>
>>> >> There are limitations to the granularity of windows bases and
>>> >> lengths. This is especially acute when using the A16 address space.
>>> >>
>>> >> To debug this, try mapping the entirety of the A16 address space
>>> >> using master_set. Then when calling read, read from offset 0x2200.
>>> >>
>>> >>> 3. When I do reads with what I believe is the correct address, I get
>>> >>> back '0xff' characters all the time, and if I do it frequently
>>> >>> enough I manage to crash the computer (with no logs on the dmesg,
>>> >>> and reboot needed with a forced fsck).  I am now trying to probe the
>>> >>> kernel module adding print statements, and trying to build it out of tree.
>>> >>>
>>> >>
>>> >> There was a bug when err_chk was set a while back, if you are running
>>> >> an older kernel you may be hitting that. It stores errors, but in
>>> >> some situations doesn't read them and clear them in time leading to
>>> >> memory exhaustion...
>>> >>
>>> >>
>>> >>> Cheers,
>>> >>> Maurice
>>> >>>
>>> >>> PS: Here is what I get when I do an 'lspci -v':
>>> >>>
>>> >>> 03:00.0 PCI bridge: PLX Technology, Inc. PEX 8114 PCI
>>> >>> Express-to-PCI/PCI-X Bridge (rev bd) (prog-if 00 [Normal decode])
>>> >>>          Physical Slot: 0
>>> >>>          Flags: bus master, fast devsel, latency 0
>>> >>>          Memory at d4000000 (32-bit, non-prefetchable) [size=8K]
>>> >>>          Bus: primary=03, secondary=04, subordinate=04, sec-latency=64
>>> >>>          Memory behind bridge: d0000000-d3ffffff
>>> >>>          Capabilities: <access denied>
>>> >>>
>>> >>> 04:04.0 Bridge: Tundra Semiconductor Corp. Tsi148 [Tempe] (rev 01)
>>> >>>          Subsystem: Tundra Semiconductor Corp. Device 0000
>>> >>>          Flags: bus master, 66MHz, medium devsel, latency 32, IRQ 16
>>> >>>          Memory at d0000000 (64-bit, non-prefetchable) [size=4K]
>>> >>>          Capabilities: <access denied>
>>> >>>          Kernel driver in use: vme_tsi148
>>> >>>
>>> >>
>>> >> The reads don't occur through the PCI bars (nasty), so you will need
>>> >> to find out what PCI addresses the windows are being mapped to and
>>> >> confirm they are in the d0000000-d3ffffff window. Without knowing
>>> >> much more about your system (and I don't think you've even told me
>>> >> what SBC you are using) there's a limit to how much I can help I'm afraid.
>>> >>
>>> >> Hope that helps,
>>> >>
>>> >> Martyn
>>> >>
>>> >>
>>> >>> On Wed, Jul 16, 2014 at 12:47 AM, Martyn Welch
>>> <martyn.welch at ge.com>
>>> >>> wrote:
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> On 14/07/14 19:29, Maurice Moss wrote:
>>> >>>>>
>>> >>>>>
>>> >>>>> Hi all,
>>> >>>>>
>>> >>>>> I have updated my Linux Kernel to the latest.  I am on Debian
>>> >>>>> 64bit 3.15.5.  I issue the following Kernel command line, and the
>>> >>>>> vme_user module seems to load correctly, however the vme bus is
>>> >>>>> neither mounted on /dev nor /proc.
>>> >>>>>
>>> >>>>
>>> >>>> Just to make sure, you're looking under /dev/bus/vme?
>>> >>>>
>>> >>>>
>>> >>>>> I was earlier using a 3.2 debian 32bit and managed to mount the
>>> >>>>> vme bus by following the exact same procedure of rebuilding the
>>> >>>>> kernel with vme_user module.  Any help is appreciated.  Here is
>>> >>>>> what I see on dmesg.
>>> >>>>>
>>> >>>>> [    0.000000] Kernel command line:
>>> >>>>> BOOT_IMAGE=/boot/vmlinuz-3.15.5-vme
>>> >>>>> root=UUID=4cdc2e84-9fbc-471c-9eb4-fde8f0b1ce96 ro
>>> vme_user.bus=0
>>> >>>>> vme_tsi148.err_chk=1 quiet
>>> >>>>> [    1.754278] vme_user: VME User Space Access Driver
>>> >>>>> [    1.754695] vme_tsi148 0000:04:04.0: Board is the VME system
>>> >>>>> controller
>>> >>>>> [    1.754700] vme_tsi148 0000:04:04.0: VME geographical address is 0
>>> >>>>> [    1.754704] vme_tsi148 0000:04:04.0: VME Write and flush and error
>>> >>>>> check is enabled
>>> >>>>> [    1.754942] vme_tsi148 0000:04:04.0: CR/CSR Offset: 0
>>> >>>>> [    1.754948] vme_tsi148 0000:04:04.0: Enabling CR/CSR space
>>> >>>>>
>>> >>>>> Cheers!
>>> >>>>>
>>> >>>>
>>> >>>> It's unfortunately going to take me a while to get everything
>>> >>>> together to take a look, some of my old installs I've been eeking
>>> >>>> along for a while to do adhoc VME tests are now broken :-(
>>> >>>>
>>> >>>> Martyn
>>> >>>>
>>> >>>>
>>> >>>>> On Thu, Jul 3, 2014 at 8:18 AM, Maurice Moss
>>> >>>>> <eightplusclub at gmail.com>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> Martyn,
>>> >>>>>>
>>> >>>>>> OK.  I feel like I am not clear.  The kernel command line has a
>>> >>>>>> space, but the line here in the email doesn't (I don't know how
>>> >>>>>> that happened).  I am still stuck with the same issue.
>>> >>>>>>
>>> >>>>>> Sorry for all the confusion
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Thu, Jul 3, 2014 at 8:15 AM, Maurice Moss
>>> >>>>>> <eightplusclub at gmail.com>
>>> >>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> Yes, copy and paste issue, I had double checked that right after
>>> >>>>>>> I sent you the mail.  Sorry!!
>>> >>>>>>>
>>> >>>>>>> On Thu, Jul 3, 2014 at 3:47 AM, Martyn Welch
>>> >>>>>>> <martyn.welch at ge.com>
>>> >>>>>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> On 03/07/14 00:47, Maurice Moss wrote:
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> I upgraded to linux kernel 3.14.9 (on Fedora).  Re-compiled
>>> >>>>>>>>> the kernel with the vme support etc.  I now get the below in
>>> >>>>>>>>> my log, and don't see any vme related files in /dev !!
>>> >>>>>>>>>
>>> >>>>>>>>> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-
>>> 3.14.9
>>> >>>>>>>>> root=UUID=aee6e594-4be8-46d4-abe6-7c054ef239b0 ro
>>> >>>>>>>>> vconsole.font=latarcyrheb-sun16
>>> >>>>>>>>> vme_user.bus=0vme_tsi148.err_chk=1
>>> >>>>>>>>> rhgb quiet
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> Unless this is a copy and paste issue, you seem to be missing a
>>> >>>>>>>> space between "vme_user.bus=0" and "vme_tsi148.err_chk=1".
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> Martyn
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>> Martyn Welch (Lead Software Engineer)  | Registered in England
>>> >>>>>>>> and Wales
>>> >>>>>>>> GE Intelligent Platforms               | (3828642) at 100 Barbirolli
>>> >>>>>>>> Square
>>> >>>>>>>> T +44(0)1327322748                     | Manchester, M2 3AB
>>> >>>>>>>> E martyn.welch at ge.com                  | VAT:GB 927559189
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Martyn Welch (Lead Software Engineer)  | Registered in England and
>>> Wales
>>> >>>> GE Intelligent Platforms               | (3828642) at 100 Barbirolli
>>> >>>> Square
>>> >>>> T +44(0)1327322748                     | Manchester, M2 3AB
>>> >>>> E martyn.welch at ge.com                  | VAT:GB 927559189
>>> >>
>>> >>
>>> >> --
>>> >> Martyn Welch (Lead Software Engineer)  | Registered in England and
>>> Wales
>>> >> GE Intelligent Platforms               | (3828642) at 100 Barbirolli Square
>>> >> T +44(0)1327322748                     | Manchester, M2 3AB
>>> >> E martyn.welch at ge.com                  | VAT:GB 927559189


More information about the devel mailing list