Visualisation Using Paraview of Washing Machine Results

Objective: Describe the methods used to import data into paraview

Paraview is a powerful interactive tool for data visualisation and analysis it is developed using the VTK toolkit. Our objective is to rapidly get our data into the visualisation platform and to explore that data se, gain new insights.

Data formats for paraview VTK, VTU, h5
Develop custom interface

There is a number of approaches we can adopt, if our data is not in a format which can be ported into Paraview then it will be necessary to develop our own converter. Paraview supports many file formats ( https://www.paraview.org/Wiki/ParaView/Data_formats ). Examples include the hdf format and the VTK format.

Example converters which were tested include implementations using Matlab and these have been published on Matlab Central, for example:

Implementations which were tested included codes in both Matlab and IDL.

WriteToVTK

The method which was most successful involved converting the output of the modelling code (SMAUG and SAC) into hdf5 format.

Converters were easily developed using Matlab e.g. see .

Method

For the the visualisations developed here a number of steps were followed.

Develop a visualisation using the Paraview GUI
Generate a python script
Run a sequence of visualisations using a task array
Run Paraview using a docker container implementation

The data set being visualised consisted of approximately 1000 separate files each containing the data for an MHD run for one a single time step. For the 128x128x128 grids considered here the file size was

218MB for and hff5 format file
268MB for a SAC3d output format

As well as a recognised standard and an efficient file format for read/write the hdf5 format is 20% smaller… bonus!

Conveniently Paraview provides a number of readers for hdf5. The import method has already been described in an earlier post Importing MHD Simulation Data into Paraview.

Paraview State Scripts

As described earlier, Paraview provides a range of file loaders and filters for loading, transformation and visualisation of processed data. The ease with which this can be achieved greatly enhances our ability to explore a data sets view them in a variety of ways and to gain insights.

One of the first steps is to save and reload the state of paraview at a particular time. The paraview state can be saved either in Paraview’s own xml based format or as a python script which can be rerun using the pvpython command. When saving using this method it is possible to select which properties will be saved ( modified, visible or all properties).

stream_contour_extract_slices_magslices_dualview

Example below includes generated python script which we can run using pvpython

The visualisation below revealed an interesting kinking effect of the magnetic flux tube.the decision was made to investigate the evolution of this effect over time. Given the size of the full data set, it was impractical to generate a significant number of images on a personal workstation. This was the point at which the pvpython task became useful. to make use of it some modifications were made.

The image creation part of the script was written into a create image function

def createimage(i,renderView1):

    id=1000+i*1000
    fnameroot='/Users/xxxxx/proj/washmc-data/uni6/h5/washmc_'
    ifnameroot='/Users/xxxxx/proj/washmc-data/uni6/images/washmc_'
    
    fname=fnameroot+str(id)+'.h5'
    ifname=ifnameroot+str(id)+'.jpg'

    # ----------------------------------------------------------------
    # setup the data processing pipelines
    # ----------------------------------------------------------------    
    # create a new 'VisItPixieReader'
    washmc_h5 = VisItPixieReader(FileName=fname)
    washmc_h5.Meshes = ['mesh_128x128x128']
    washmc_h5.CellArrays = ['data/grid_0000000000/density_bg', 'data/grid_0000000000/density_pert', 'data/grid_0000000000/internal_energy_bg', 'data/grid_0000000000/internal_energy_pert', 'data/grid_0000000000/mag_field_x_bg', 'data/grid_0000000000/mag_field_x_pert', 'data/grid_0000000000/mag_field_y_bg', 'data/grid_0000000000/mag_field_y_pert', 'data/grid_0000000000/mag_field_z_bg', 'data/grid_0000000000/mag_field_z_pert', 'data/grid_0000000000/velocity_x', 'data/grid_0000000000/velocity_y', 'data/grid_0000000000/velocity_z', 'grid_left_index', 'grid_level', 'grid_parent_id', 'grid_particle_count']


.
.
.
.
.
.

The second code fragment, below, shows the loop in the main part of the routine and the loop which repeatedly calls the createimafe() routine.

# ----------------------------------------------------------------
# restore active view
SetActiveView(renderView1)
# ----------------------------------------------------------------

for i in range(0,1271): 
    #### disable automatic camera reset on 'Show'
    paraview.simple._DisableFirstRenderCameraReset()
    
    # get the material library
    materialLibrary1 = GetMaterialLibrary()
    id=1000+i*20000
    fnameroot='/Users/xxxxxxx/proj/washmc-data/uni6/h5/washmc_'
    ifnameroot='/Users/xxxxxxx/proj/washmc-data/uni6/images/washmc_'
    
    # Create a new 'Render View'
    renderView1 = CreateView('RenderView')
    renderView1.ViewSize = [1942, 1412]
    renderView1.AxesGrid = 'GridAxes3DActor'
    renderView1.CenterOfRotation = [32.0, 64.0, 64.0]
    renderView1.StereoType = 'Crystal Eyes'
    renderView1.CameraPosition = [239.76251878719324, 0.640916434808301, 308.3771165833229]
    renderView1.CameraFocalPoint = [4.065694001785766, 72.51882254031378, 31.142749358984677]
    renderView1.CameraViewUp = [0.705407579263351, -0.24797493532912449, -0.6640094717444442]
    renderView1.CameraFocalDisk = 1.0
    renderView1.CameraParallelScale = 96.0
    renderView1.BackEnd = 'OSPRay raycaster'
    renderView1.OSPRayMaterialLibrary = materialLibrary1
    
    #SetActiveView(None)    
    # ----------------------------------------------------------------
    # setup view layouts
    # ----------------------------------------------------------------    
    # create new layout object 'Layout #1'
    layout1 = CreateLayout(name='Layout #1')
    layout1.AssignView(0, renderView1)    
    # ----------------------------------------------------------------
    # restore active view
    #SetActiveView(renderView1)
    # ----------------------------------------------------------------
    
    createimage(i, renderView1) 
'
'
.
.
.

The resulting movie (generated using QuickTime (or ffmpeg is shown below:

The task to generate the images was run using a singularity container ( see Singularity on ShARC ) the singularity container was built using a paraview docker container.

This was highly effective. The procedure which worked was as follows:

Run a conversion script generating .h5 (or even .vtu) files which can be read into Paraview
Copy a small number of files from remote system to local desktop
Investigate sample using Paraview and save states
For the selected scenes of interest generate the python script and convert so that it can generate multiple frames
Run the task using Paraview in the singularity container

Experiences Using pvserver

Methods which were used for the visualisation included the following

Mounting the remote storage on the desktop system
Running paraview using Oracle desktop, application similar to x2go or openondemand
Using a VirtualGL type client e.g. with tigerVNC

Another method which is highly effective is the use of a paraview client server system in which we can run paraview on a local desktop and the paraview server processes the data on the remote generating a rendered image. This proved problematic and the solution was not fully resolved, we look forward to resolving this soon!

Guidelines for running and using pvserver are at Running ParaView in Client-Server Mode.

Future Developments

Custom loader plugin e.g. see:

Useful Links

Pictures with hyperlinks in markdown

embedding moview

2d view showing mutually perpendicular slices and the B field magnitude, lower figure shows a horizontal slice at the bae of the model

Copy the link to share and you will have something like https://drive.google.com/file/d//view?usp=sharing

Copy the to make a link like this: https://drive.google.com/uc?export=view&id=

Insert image in Markdown as ususal using the link from step 4. For example: