AD2 continuous digital acquisition speed is slow

[Flo]

Hi,

I'm using AD2 from mac using the SDK and I want to do continuous acquisition on the digital IO.

When executing my program (sample format 8bits, divider 200) I can't get much more than 500Ksamples/sec otherwise I get data lost and corruptions.

Since the AD2 is given for 100MSamples/sec and the USB2 is ~50Msamples, I expected at least 10MSamples. Is this expected or am I missing something here ?

 

Thanks,

Flo

[av_disp]

the continuation.

sample rate / num of samples/ num of loss occurrences/ num of lost samples

1MHz           10.000000            11                                       45056

1MHz           10.000000            5                                         65536

1MHz           1.000000            2                                           28672

1MHz           1.000000            1                                           4096

2MHz           1.000000            1                                           8192

2MHz           1.000000            2                                           8192

2MHz           10.000000            12                                       409600

2MHz           10.000000            7                                         40960

2MHz           10.000000            7                                         49152

2MHz           10.000000            7                                         36864

PS: num of loss occurrences -  is the number of times when cLost.value > 0, number of lost samples is += cLost.value  in terms of DigitalIn_Record.py

if the number of samples is set to 1.000.000 then no samples are lost at 2MHz but this very short period since we are talking about possibility to continuously stream data to a PC

And this is a standalone python script, in WF you will have 5 time slower sample rate (there are several posts on the forum)  I have searched the forum and saw the staff's explanation why we have such results:

a) USB bandwidth limitation. That makes no sense to me (just personal opinion) since USB 2.0 has more than  enough bandwidth to cope with those data acquisition speeds (people were asking if USB 3 would help - even USB 2 in this case is far from saturation (just looking at the numbers);

b )  The necessity to poll several tools at once  (in case of WF) and as a result the need to use USB bus for multiple transfers  in addition to stream data from AD2 to PC.  That makes no sense to me either since the  result does not depend on the number of WF tools - I can open only Logic analyzer and still not get 1MHz or so  s.r. in WF for any buffer that is longer a certain, fixed number. And yes, it works fine for short periods i.e. small set of data.

 

There is  possibility that I, somehow misunderstood something and therefore the provided results are not entirely correct. But at the moment my understanding is that one can either record a short periods of time with high (relatively because it does not come even close to 100MS/s (yes, I understand that this is an internal s.f.) sample frequency or longer periods with  lower sample frequency - low enough not to allow you to reliably capture 400 kHz SPI data exchange from WF   for 5 seconds.

Thanks.

[StuE]

@attila thanks, I actually found it after digging around for a while. Sorry to have asked a question without looking into things myself first.

[attila]

Hi @StuE

Here you can see the device configuration capabilities:

>python Device_Enumeration.py
DWF Version: b'3.10.2'
Number of Devices: 1
------------------------------
Device 0 :
        Name:'b'Analog Discovery 2'' b'SN:210321A419AA'
        Configurations:
        0. AnalogIn: 2 x 8192   AnalogOut: 2 x 4096     DigitalIn: 16 x 4096    DigitalOut: 16 x 1024
        1. AnalogIn: 2 x 16384  AnalogOut: 2 x 1024     DigitalIn: 16 x 1024    DigitalOut: 0 x 0
        2. AnalogIn: 2 x 2048   AnalogOut: 2 x 16384    DigitalIn: 0 x 0        DigitalOut: 0 x 0
        3. AnalogIn: 2 x 512    AnalogOut: 2 x 256      DigitalIn: 16 x 16384   DigitalOut: 16 x 16384
        4. AnalogIn: 2 x 8192   AnalogOut: 2 x 4096     DigitalIn: 16 x 4096    DigitalOut: 16 x 1024
        5. AnalogIn: 2 x 8192   AnalogOut: 4 x 4096     DigitalIn: 16 x 2048    DigitalOut: 8 x 256

 

[StuE]

@attila Can I ask the reason FDwfDeviceConfigopen doesn't support AnalogIn/Out?

[kojak]

I have another idea to solve slow speed - can you make a single instrument setting so it will be possible to use only one but with maximum performance and maximum memory available?

[attila]

Hi @av_disp

a,b, I'm using the released software version 3.5.4.
c, The "device firmware" is programmed by the software on first open after reset.

To have more device buffer (fifo for record) for digital in you could select the 3rd configuration.
This configuration provides 16k samples for digital in and out, but does not support analog in/out.
FDwfDeviceConfigOpen(int idxDev, int idxCfg = 3, HDWF *phdwf)

Try to close other application which could periodically block/use the system, like messaging apps.
Make sure to not to have other USB device connected to the same hub which could reduce the available USB bandwidth.

[jpeyron]

Hi @av_disp,

Sorry about the delay. I am re-tagging @attila to make sure that he sees you post.

cheers,

Jon

[av_disp]

Hello,

Is there any update?

Thanks

[av_disp]

Hello,

--------

>I use a late 2013 MacBookPro, i74850HQ, 16GB, OS X 10.11
>The DigitalIn_Record.py with 1M samples is working fine for me down to divider 26, up to ~3.8MHz.

-------

>i'm using a 2014 macbook pro with 2,5Ghz Core i7 and 16GB ram, so I guess this should not be the bottleneck...

>When executing my program (sample format 8bits, divider 200) I can't get much more than 500Ksamples/sec otherwise I get data lost and >corruptions.

-------

It's interesting that you both use similar Apple hardware and get such different results. I will try to find a MacBook but attila

could you please  tell us the following:

a) version of WF;

b ) version of FTDI driver;

c) firmware version of your AD2 (btw, is it upgradable? ) 

d) whatever else might be useful to know to be able to replicate your environment;

It would be also very interesting to know what do you get if you repeat the same test with on Windows using the latest software available on your web site - I just want to know if those numbers  (divider 26 etc) are really achievable at present time for Windows  users using WF 3.5.4 installer.

 

thanks

 

 

[jpeyron]

Hi @av_disp,

I am tagging the engineer that will be able to give you the best response to you question( @attila).

thank you,

Jon

[av_disp]

Hello,

I am observing a similar situation and pretty much sure that the problem is not in the user's hardware.

I already  spent a lot of time trying to make it work at some acceptable level but with no success. I have tried 3 different computers, WinXP, Win7 32/64, two (different high end Lenovo)  of them have SSD with write speed way faster than 100 MB/s, 24GB-32GB RAM, different USB 2 and 3 controllers, 2 different (USB 2 and USB 3) hubs, WaveForms 3.5.4, AD2 is powered by an individual 5V power supply.

I tried "record" mode from WaveForms and from standalone python script (modified DigitalIn_Record.py). The result i.e. whether or not you get a message about lost/corrupted samples depends on both the sample rate and the number of samples to acquire.

On the most powerful Lenovo (32GB RAM, SSD 450 MB/s write speed), with both USB2/3 plus with/without dock's USB 2 controller I can get the following result:

sample rate / num of samples/ num of loss occurrences/ num of lost samples

1MHz           10.000000            11                                       45056

 

 

Spoiler

Spoiler

 

 

 

[attila]

I don't know what could limit the recording rate for you.
With this data streaming there is no guaranteed rate since it can be influenced by host controller, other devices, running applications/services...

Sorry for the late reply,
Attila

[Flo]

I'm using OS X 10.11.4 and yes the device is connected directly to the Mac.

[attila]

I use a late 2013 MacBookPro, i74850HQ, 16GB, OS X 10.11
The DigitalIn_Record.py with 1M samples is working fine for me down to divider 26, up to ~3.8MHz.

Do you connect the Discovery directly to the Mac or do you use hub and share the USB line with other devices?

[Flo]

I tried with acqmodeRecord and the results are pretty much similar.

I also tried the python example with divider = 100 and I got sample lost/corrupted at this rate.

What is your mac config ? and what is the max config rate you can get without lost/corrupted samples ?

[attila]

See the DigitalIn_Record.py example.
Use acqmodeRecord for recording. I'm not sure what does the status-record return for scan acquisition.

[Flo]

Hello,

i'm using a 2014 macbook pro with 2,5Ghz Core i7 and 16GB ram, so I guess this should not be the bottleneck...

Did you get the python code running at 2Mhz continuously without getting any corrupted or lost sample ?

Here is the code I'm running, as you can see I disabled all printf and I'm no even saving any data in the loop:

	int freqDivider = 100;
    
    FDwfDeviceOpen(0, &hwHandle);
    
    //FDwfDigitalInClockSourceSet(hwHandle, DwfDigitalInClockSourceInternal);
    FDwfDigitalInAcquisitionModeSet(hwHandle, acqmodeScanShift);
    FDwfDigitalInDividerSet(hwHandle, freqDivider);
    FDwfDigitalInSampleFormatSet(hwHandle, 16);
    FDwfDigitalInTriggerSourceSet(hwHandle, trigsrcDetectorDigitalIn);
    FDwfDigitalInTriggerSet(hwHandle, 0,0, 0xFFFF, 0xFFFF);
    
    FDwfDigitalInBufferSizeInfo(hwHandle, &maxInternalBufferSize);
    FDwfDigitalInBufferSizeGet(hwHandle, &currentInternalBufferSize);
    std::cerr << "Internal buffer size, max="<< maxInternalBufferSize<<" ,current=" << currentInternalBufferSize << std::endl;
    
    FDwfDigitalInInternalClockInfo(hwHandle, &internalFreq);
    FDwfDigitalInClockSourceInfo(hwHandle, &clockSource);
    std::cerr << "Clock source="<< clockSource<<" ,Freq=" << (uint64_t)internalFreq << "Mhz ,Sampling Rate=" << (uint64_t)(internalFreq/freqDivider) << std::endl;

    
    
    FDwfDigitalInConfigure(hwHandle, false, true);
    
    std::cerr << "Acquiring..." << std::endl;
    while(1)
    {
        FDwfDigitalInStatus(hwHandle, true, &deviceState);
        
        FDwfDigitalInStatusRecord(hwHandle, &available, &lost, &corrupted);
        
        if(available == 0)
        {
            continue;
        }
        
        if(lost != 0)
        {
            std::cerr << "Lost samples !!" << std::endl;
        }
        
        if(corrupted != 0)
        {
            std::cerr << "Corrupted samples !!" << std::endl;
        }
        
        if(available != 0)
        {
            FDwfDigitalInStatusData(hwHandle, tmp, available);
            //std::cerr << "Data available " << available << "bytes" << std::endl;
            //out.write((char *)tmp, available);
        }
    }

 

[attila]

Hello,

The maximum sustainable continuous recording rate depends on many system factors.
On my MacBook the digitalin_record Python example code is running above 2MHz.

The achievable USB transfer rate is around 30-40 MBps.
For the present, the 8bit format is only for usage convenience, always 16bit samples are read from the device. 
The communication with the device is optimized for low latency, to serve multiple instruments (scope, AWGs...). The recording is solved in transfer chunks and this reduces the streaming rate.