benl

Members
  • Content Count

    13
  • Joined

  • Last visited

  • Days Won

    1

benl last won the day on May 1

benl had the most liked content!

About benl

  • Rank
    Member

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. Thanks for the update @JColvin; obviously not what we'd like to hear in so far as lack of resources behind the product but the communications is appreciated. TBH, the dlog-utils code is... not great. The majority of the code is in type conversion and formatting (i.e. not germane to the actual processing of the data); I'm not surprised to hear it's problematic in updating it for OpenLogger as hard-coded assumptions on the data header abound (e.g. endianness; I presume the author is banking on that never changing, which may well be the case but it is in the format spec). As a reference implementation it hides the important data structure information in amongst language-specific type gymnastics. In contrast the Kaitai Struct approach removes all of that, and puts the data format front and centre, is trivially extensible (you update the struct definition and rebuild the library, done), and works "everywhere". If it were my decision I'd dump the current dlog-utils and start again based on Kaitai Struct, the result would be: a proper definition of the data format (rather than users having to reverse engineer the cpp code and troll the forums) a couple of dozen of lines of code for the reference Digilent implementation and most importantly would be useful/portable in any language/environment that Kaitai Struct supports (C++/STL, C#, Go, Java, JavaScript, Lua, Perl, PHP, Python, Ruby) As an example, what is implemented in nearly 180 LOC dlog-utils.cpp is under a dozen lines in the dlog-utils-portable Python example (`dlog = Dlog.from_file(args.inputfile)` followed by a `write_csv`), with far greater flexibility in terms of handling future variations on data formats, and better output formatting 🙂 Given that Digilent have very limited resources for this project it's important they're used wisely, switching to Kaitai Struct is easily the best bang for buck we can ask for. (BTW, it might sound like I'm a shill for Kaitai Struct - nope, I'm just a satisfied user and first discovered it when writing dlog-utils-portable... I once wrote code to process structured binary data in the same way as dlog-utils, but I've now seen the light 🙂
  2. The example I posted would work for Linux or Mac with "common" tools installed. As to Windows... can't really help much there. git's not part of Python, it's used for managing code; you can achieve the same end result here by downloading the ZIP from https://github.com/bdlow/dlog-utils-portable/archive/master.zip and unzipping to a folder. Virtual environment support is a standard part of Python 3; you can skip that if you like but without virtual environments eventually your Python installation will end up like this: https://xkcd.com/1987/ Ah, of course, in Windows `activate` is a batch script not a shell script: https://www.techcoil.com/blog/how-to-create-a-python-3-virtual-environment-in-windows-10/
  3. @sgrobler: I've tweaked the project to hopefully make it simpler to install the dependencies (Kaitai Struct v0.9). If you have Python 3 installed, it should simply be a matter of: % git clone https://github.com/bdlow/dlog-utils-portable.git Cloning into 'dlog-utils-portable'... remote: Enumerating objects: 26, done. remote: Counting objects: 100% (26/26), done. remote: Compressing objects: 100% (19/19), done. remote: Total 26 (delta 8), reused 20 (delta 5), pack-reused 0 Unpacking objects: 100% (26/26), done. % cd dlog-utils-portable dlog-utils-portable% ls LICENSE dlog.ksy dlogcsv.py requirements.txt README.md dlog.py examples # set up and activate a Python virtual environment: dlog-utils-portable% python3 -m venv .venv dlog-utils-portable% . .venv/bin/activate # install the 0.9 runtime in the virtual environment: (.venv) dlog-utils-portable% pip install -e 'git+https://github.com/kaitai-io/kaitai_struct_python_runtime.git@0e3f6e0#egg=kaitaistruct' Obtaining kaitaistruct from git+https://github.com/kaitai-io/kaitai_struct_python_runtime.git@0e3f6e0#egg=kaitaistruct Cloning https://github.com/kaitai-io/kaitai_struct_python_runtime.git (to revision 0e3f6e0) to ./.venv/src/kaitaistruct Did not find branch or tag '0e3f6e0', assuming revision or ref. Installing collected packages: kaitaistruct Running setup.py develop for kaitaistruct Successfully installed kaitaistruct # run it! (.venv) dlog-utils-portable% ./dlogcsv.py ./examples/openlogger.dlog > openlogger.csv Header Information log format: openlogger stop reason: normal number of samples: 27097 voltage units: mV sample rate: 10E+3 Sa/s delay: 0 s number of channels: 3 channel map: [1, 2, 3] (.venv) dlog-utils-portable% less openlogger.csv
  4. @Digilent: it would be awesome if you could review the KSY definition, or publish an official one.
  5. Here's an OpenScope and OpenLogger log file utility that can export to CSV (TODO: JSON). It's written in Python, but the important part (the log file parsing) is portable across about a dozen other languages. https://github.com/bdlow/dlog-utils-portable I've used a combination of the data structure definitions in the Digilent dlog-utils project, some information posted in this forum on the OpenScope data format, and the application of educated guesswork. IMHO this is an improvement over the current version of dlog-utils, which has various log file parameters hard-coded within even though these parameters are specified within the log file header. It's also more portable (Python 3), smaller and of course handles both log types. This work was made possible through the use of the awesome Kaitai Struct project to define the log file structure and automatically generate a Python parsing library from that. Kaitai handles all the details of the log formatting including endianness and data types, and presents the log via a very easy to use Python class. The web IDE makes it very easy to poke around the binary log file. Want to use something other than Python? All you need's the `.ksy` file and you're almost there for any of the languages that Kaitai supports: C++/STL, C#, Go, Java, JavaScript, Lua, Perl, PHP, Python and Ruby. I've a JSON exporting version almost ready, though I probably ought to add some tests first. @Digilent: can you expand on what the channel map is meant to represent? I had presumed it was the channel number to sample index mapping, e.g. if I recorded channels 2, 3 and 7 in the OpenLogger then I'd have expected the data to have three channels in each sample; and the channel map to look like [2, 3, 7, 0, ...]. However my OpenLogger generates log files with a channel map [1, 2, 3]. Bug?
  6. Oh, absolutely agree / understand that some form of decimation and interpolation is required to display the waveform (though my DSP is getting pretty rusty). The current implementation is clearly inadequate to the point of not being useful: the waveform displayed when zoomed out is a very inaccurate representation of the signal. You should at least be able to get a somewhat accurate view of the signal's envelope. The same goes for the buffer view at the top; at present it's not useful to inform on much beyond the relative position of the display within the buffer - ideally you'd be able to use it to pick out envelope changes and transients (the latter would certainly benefit from a peak-detection mechanism). In a nutshell, it'd be useful if WFL presented the signal in a similar fashion as the same data on a deep-memory oscilloscope. Ta, Ben
  7. What does the host Agent application do? I've discovered it's not needed to run Waveforms Live when connecting to an OpenLogger via wifi; I presume it's only needed when using USB connectivity? (i.e. it doesn't presently serve any useful purpose on macOS/Linux) for the OpenLogger.
  8. G'Day Andrew, Uh, so there's no way to view the recorded data? Oops. (<flashback to the Seinfeld "most important part" scene> :-) Obviously it'd be great to load / replay the recorded data back into WFL, the interface is pretty neat and I'm a fan of the browser-as-an-app-platform. In the meantime, yes, can you please provide the header? (btw, when's all this going up on github?) Ta, Ben
  9. There is some pretty messy aliasing happening in Waveforms Live's buffer view and also the main chart. For example, below is the AWG connected to channel 1, zoomed out. It looks fine on the main chart when zoomed in (though the buffer view remains full of alias artefacts). It would be great if the signal envelope was represented properly in both the buffer view and the chart, regardless of zoom.
  10. So I've successfully logged some data to SD card on the OpenLogger MZ (0.1807.0); I can download the .log file and though I've yet to try it I see the dlog utility can be used to spit out a CSV file. How can you use WFL to view the logged data?
  11. Thanks JColvin; I was able to reduce / eliminate the "fall behind" errors by reducing the sampling rate (should have tried this earlier!); I found that up to 100kS/s seems to work reliably in terms of streaming data; however interacting with the AWG at anything over about 20kS/s causes ""Could not set AWG parameters" errors. Hmm, I just noticed the official specs / FAQ states streaming over wifi at 10kS/s. So I guess this is working as expected. Also I found the firmware version info is reported in the 'Settings' page you referred to; as is the Waveforms Live version. Re. logging: setting the log to the console logs a boatload of debug level logs, not errors. Indeed, I can't actually find the errors reported via the 'toast' notifications logged to console at all in the logs when enabled.
  12. Have just started playing with the OpenLogger that just arrived. Did a firmware update and calibration via a Windows 10 VM on a macOS machine, and connected the OpenLogger to wifi, all went fine. Is wifi meant to be working? If so, consider the following as a bug report. If not, then what's the ETA looking like? OL firmware version: whatever the latest is (how can I determine this from a running device? the UI should have some method of viewing the connected device's firmware version (e.g. in device manager?)) env: Chrome Version 73.0.3683.103 (Official Build) (64-bit); macOS 10.14.4 (18E226) However when connected to the OpenLogger via wifi I see frequent errors, including - "Error Occurred Starting Or Running Acquisition" when interacting with the timebase and amplitude (i.e. mouse zooming/scrolling) - "Could not set AWG parameters" when interacting with the AWG controls, changing modes There doesn't seem to be anything in the browser dev tools console when these occur. (the console does show a bunch of warnings re. 't' plugin and a CORS policy error just on loading the waveformslive.com page, but there's nothing additional when the acquisition error occurs). Repro: 0. OpenLogger connected to wifi 1. go to http://waveformslive.com/ 2. start streaming 3. interact with the UI - scroll back and forth, zoom in/out 4. in very short order one of the above error messages pops up; at this point the UI becomes unresponsive and a page reload / device reconnect is required Occasionally the device will get itself so confused it won't connect at all (WFL reports "No response received"), and a OL board reset is required. PS: there needs to be some sort of error log mechanism, it's a real pain to try and capture the errors via toasts....