58169f6 | Josef Stumpfegger | 08 June 2022, 11:13:43 UTC | Trying out non atomic increment. | 08 June 2022, 11:13:43 UTC |
67afe0e | Josef Stumpfegger | 07 June 2022, 17:19:04 UTC | A lot of bugfixes... | 07 June 2022, 17:19:04 UTC |
2c432b3 | Josef Stumpfegger | 04 June 2022, 08:31:19 UTC | Axis scaling now works, added activation to the cpu implementations, recalculation now also works. | 04 June 2022, 08:31:19 UTC |
5d802cd | Josef Stumpfegger | 03 June 2022, 17:40:01 UTC | Huge improvement for hierarchy creation, added index activation and axis scaling. | 03 June 2022, 17:40:01 UTC |
f7c44cb | Josef Stumpfegger | 02 June 2022, 17:44:36 UTC | Found better compression reasons in the previos id files. Compression time is very slow: test for normla vector list with following insertion into roaring bitmaps. | 02 June 2022, 17:44:36 UTC |
d94d9ce | Josef Stumpfegger | 01 June 2022, 17:04:54 UTC | Added cpu and gpu code for rle decoding. Still has to be fully implemented and integrated. | 01 June 2022, 17:04:54 UTC |
c1f3664 | Josef Stumpfegger | 31 May 2022, 17:21:38 UTC | Added first cpu side things for Huffman decoding. Still missing pipeline setup and other things. | 31 May 2022, 17:21:38 UTC |
ab4e336 | Josef Stumpfegger | 31 May 2022, 16:42:04 UTC | Added huffman decode transpose shader | 31 May 2022, 16:42:04 UTC |
bc14a55 | Josef Stumpfegger | 31 May 2022, 14:07:10 UTC | Added huffman decoding gpu code | 31 May 2022, 14:07:10 UTC |
2edca73 | Josef Stumpfegger | 27 May 2022, 10:49:28 UTC | Huffman table cpu side mostly implemented. GPU side missing. | 27 May 2022, 10:49:28 UTC |
0088689 | Josef Stumpfegger | 27 May 2022, 08:48:02 UTC | Started to implement vkCompress to be able to decompress the datablocks on the fly. | 27 May 2022, 08:48:02 UTC |
840b9b3 | Josef Stumpfegger | 26 May 2022, 17:44:26 UTC | lineCountAll functions now without problems, bug when wanting to close the application fixed. For 6.6 million datapoints coutn for all attributes takes 340ms. | 26 May 2022, 17:44:26 UTC |
00c0454 | Josef Stumpfegger | 26 May 2022, 13:20:45 UTC | Fixed bug in data.hpp, added new subsampling strategy for laoding (random sampling). | 26 May 2022, 13:20:45 UTC |
4993df1 | Josef Stumpfegger | 25 May 2022, 17:04:32 UTC | GPU pipelines are connected and are working. All once compute pipeline currently under development. | 25 May 2022, 17:04:32 UTC |
4d0b390 | Josef Stumpfegger | 24 May 2022, 20:03:22 UTC | Added dropdown menu to change counting strategy. | 24 May 2022, 20:03:22 UTC |
a5fac25 | Josef Stumpfegger | 24 May 2022, 19:45:52 UTC | Test dataset with 10 mio datapoints counted with cpu approach now works including rendering. | 24 May 2022, 19:45:52 UTC |
3a50a04 | Josef Stumpfegger | 23 May 2022, 19:56:09 UTC | Compilable adoption of indbinmanager to PCViewer.cpp. Started creation of the compression method to produce the correct format. | 23 May 2022, 19:56:09 UTC |
d8fbc13 | Josef Stumpfegger | 23 May 2022, 13:35:33 UTC | Added safety checks at pipelines plus freeing of command buffers. Project compilable again. | 23 May 2022, 13:35:33 UTC |
d871212 | Josef Stumpfegger | 23 May 2022, 13:28:35 UTC | Added renderer ot the IndBinManager. Next: add IndBinManager to drawlist. | 23 May 2022, 13:28:35 UTC |
8c0d365 | Josef Stumpfegger | 23 May 2022, 12:27:27 UTC | Added gpu pairwise counting pipelines to the bin manager. Render counter not yet completed (missing float to uint conversion pipeline). | 23 May 2022, 12:27:27 UTC |
9859bd6 | Josef Stumpfegger | 23 May 2022, 08:31:15 UTC | Cpu counting adopted hooked up to IndBinManager. | 23 May 2022, 08:31:15 UTC |
29824d7 | Josef Stumpfegger | 19 May 2022, 19:46:14 UTC | Added automatic half conversion, cpu counting added. Other connections are still missing. | 19 May 2022, 19:46:14 UTC |
d73a04a | Josef Stumpfegger | 18 May 2022, 17:21:49 UTC | Implemented loading and conversion of 1d indexlists. Compilable, not yet debugged. Next: Data loader for compressed and uncompressed formats, GpuDecompression. | 18 May 2022, 17:21:49 UTC |
12bd11b | Josef Stumpfegger | 18 May 2022, 14:10:06 UTC | Corrected benchmarktests for lineCounter for proper non partitioned timings. | 18 May 2022, 14:10:06 UTC |
f0bf245 | Josef Stumpfegger | 17 May 2022, 18:23:22 UTC | Further implemented counting. | 17 May 2022, 18:23:22 UTC |
83c3bcf | Josef Stumpfegger | 12 May 2022, 18:12:54 UTC | Started implementation of general load manager for 2d binning and rendering. | 12 May 2022, 18:12:54 UTC |
4db50dd | Josef Stumpfegger | 09 May 2022, 18:06:04 UTC | Added renderer for bin counts. Compilable but not yet fully tested with true data. | 09 May 2022, 18:06:04 UTC |
f048783 | Josef Stumpfegger | 28 April 2022, 15:18:38 UTC | Fixed compile warnigns. | 28 April 2022, 15:18:38 UTC |
ffec189 | Josef Stumpfegger | 27 April 2022, 18:37:31 UTC | Added extra test file for faster compilation when changing tests, tried faster compute shader, added roar bitmap tests for intersection calculation, added range class for more beatiful irange iteration, commented out unnecessary things in hierarchy bin manager and tested real world performance of roaring bitmaps on over half a billion datapoints. | 27 April 2022, 18:37:31 UTC |
672193b | Josef Stumpfegger | 26 April 2022, 13:30:44 UTC | Test code for roaring bitmaps: Avg compression ratio for real world data was at 1:~32 | 26 April 2022, 13:30:44 UTC |
17c957f | Josef Stumpfegger | 13 April 2022, 13:50:58 UTC | Added roaring bitmaps for byte size tests. | 13 April 2022, 13:50:58 UTC |
a74e7cd | Josef Stumpfegger | 12 April 2022, 16:20:32 UTC | Code futher implemented for timings | 12 April 2022, 16:20:32 UTC |
bc49d9c | Josef Stumpfegger | 08 April 2022, 09:24:10 UTC | Added a cpu counting to the pcviewer for performance comparison. | 08 April 2022, 09:24:10 UTC |
3b62f44 | Josef Stumpfegger | 07 April 2022, 18:53:17 UTC | Added rendering counting pipeline. For 2^30 datapoints 800 ms are needed for computation. | 07 April 2022, 18:53:17 UTC |
b3c3a01 | Josef Stumpfegger | 07 April 2022, 15:54:29 UTC | 2^30 points test with good spread -> 1.4s for counting between two attributes. | 07 April 2022, 15:54:29 UTC |
0c8adc9 | Josef Stumpfegger | 07 April 2022, 15:18:06 UTC | First implementation using compute shader for line counting done. Does not work for very large data sizes. | 07 April 2022, 15:18:06 UTC |
ce5f744 | Josef Stumpfegger | 07 April 2022, 08:24:52 UTC | Started implementation of counting pipeline. | 07 April 2022, 08:24:52 UTC |
f05b476 | Josef Stumpfegger | 01 April 2022, 14:00:57 UTC | Better stopwatch, better combination calculation. | 01 April 2022, 14:00:57 UTC |
058b4a3 | Josef Stumpfegger | 01 April 2022, 09:02:20 UTC | Working implementation for cluster counting. Will get a runtime improvement. | 01 April 2022, 09:02:20 UTC |
71c632d | Josef Stumpfegger | 28 March 2022, 08:20:11 UTC | Fixed build for ubuntu. | 28 March 2022, 08:20:11 UTC |
7f41046 | Josef Stumpfegger | 27 March 2022, 17:27:25 UTC | Further implemented counting active lines. | 27 March 2022, 17:27:25 UTC |
414a80f | Josef Stumpfegger | 22 March 2022, 18:57:00 UTC | Started to implement loading and conversion to drawable line instances. | 22 March 2022, 18:57:00 UTC |
e12fb1b | Josef Stumpfegger | 22 March 2022, 14:07:55 UTC | Started implementation of bin loading and conversion manager. | 22 March 2022, 14:07:55 UTC |
b32ea8a | Josef Stumpfegger | 22 March 2022, 13:38:34 UTC | 1d cluster indices implementation done. | 22 March 2022, 13:38:34 UTC |
7357030 | Josef Stumpfegger | 17 March 2022, 18:54:05 UTC | Further implemented rendering. Data conversion to correct format done, now rendering with indexbuffercreation can be implemented. Only testing is missing then. | 17 March 2022, 18:54:05 UTC |
7164438 | Josef Stumpfegger | 17 March 2022, 10:07:29 UTC | Initial commit for rendering pipeline. Runs without errors. | 17 March 2022, 10:07:29 UTC |
93d98cb | Josef Stumpfegger | 16 March 2022, 20:09:42 UTC | Loading done and debugged. | 16 March 2022, 20:09:42 UTC |
b7bab3d | Josef Stumpfegger | 16 March 2022, 09:33:33 UTC | First implementation for loading new hierarchys done. Still has to be debugged. | 16 March 2022, 09:33:33 UTC |
c7baadd | Josef Stumpfegger | 15 March 2022, 20:11:44 UTC | Started implementing HierarchyLoadManager to load the subdimension clusters. | 15 March 2022, 20:11:44 UTC |
7ef5216 | Josef Stumpfegger | 14 March 2022, 18:56:57 UTC | Disabled stopwatches | 14 March 2022, 18:56:57 UTC |
fe36de8 | Josef Stumpfegger | 14 March 2022, 18:37:48 UTC | Tried to do multithreading speedup. | 14 March 2022, 18:37:48 UTC |
3f3568b | Josef Stumpfegger | 11 March 2022, 19:22:24 UTC | Starte implementation of nd clustering. | 11 March 2022, 19:22:24 UTC |
159c74b | Josef Stumpfegger | 10 March 2022, 18:36:11 UTC | Inbetween commit. | 10 March 2022, 18:36:11 UTC |
4175958 | Josef Stumpfegger | 10 March 2022, 13:37:44 UTC | Started implementation of ndHierarchy creation. Permutation creation for n dimensions done. | 10 March 2022, 13:37:44 UTC |
b7c2ee8 | Josef Stumpfegger | 08 March 2022, 09:51:39 UTC | Finished implementation and timing of hashLeaderNode. Hash vector leader node is superior to hashLeaderNode. Timings for hash vector leader node was commented out for possible future tests. | 08 March 2022, 09:51:39 UTC |
5907d75 | Josef Stumpfegger | 04 March 2022, 17:17:10 UTC | Added hash map accelerated hierarchy creation. Timing watches are still included in this commit, should be removed. Hash map acceleration is faster. Implementation for hash map accelerated reference pointer begun, still has to be finished and timed. | 04 March 2022, 17:17:10 UTC |
bc45f2c | Josef Stumpfegger | 03 March 2022, 12:15:53 UTC | Fixed bundle reloading. | 03 March 2022, 12:15:53 UTC |
257929e | Josef Stumpfegger | 03 March 2022, 09:40:22 UTC | Implemented non compressing pipeline for comparison. | 03 March 2022, 09:40:22 UTC |
52ef13c | Josef Stumpfegger | 02 March 2022, 20:44:54 UTC | Hierarchy creation and compression fixed, hierarchy loading adapted. | 02 March 2022, 20:44:54 UTC |
67a5789 | Josef Stumpfegger | 25 February 2022, 18:56:30 UTC | Hierarchy import manager adopted to be able to load the new hierarchy files. | 25 February 2022, 18:56:30 UTC |
8cff003 | Josef Stumpfegger | 25 February 2022, 10:09:59 UTC | Started to implement decompression of bundled data. | 25 February 2022, 10:09:59 UTC |
b79bbed | Josef Stumpfegger | 25 February 2022, 09:16:28 UTC | Bundled compression done. | 25 February 2022, 09:16:28 UTC |
20ad471 | Josef Stumpfegger | 24 February 2022, 20:25:33 UTC | Added the ability to get all needed cache nodes out of the hierarchy in a sinlge pass including caching these file in a single pass. | 24 February 2022, 20:25:33 UTC |
844b1a7 | Josef Stumpfegger | 23 February 2022, 19:23:29 UTC | Started to add more advanced caching methods with an cache manager interface to be able to exchange the caching backend. Also added a new leaders clustering approach, namely vector leader node which has only vectors to hold leaders information for more efficiency. In HierarchyCreation the new methods for the new caching manager to compress the temporary files has been begun to be implemented. | 23 February 2022, 19:23:29 UTC |
971faac | Josef Stumpfegger | 17 February 2022, 13:16:55 UTC | Compilable under windows, still has some memleaks. | 17 February 2022, 13:16:55 UTC |
6e16e54 | Josef Stumpfegger | 17 February 2022, 08:29:20 UTC | Caching bug solved. | 17 February 2022, 08:29:20 UTC |
a7d064d | Josef Stumpfegger | 16 February 2022, 18:00:05 UTC | A lot of bugfixes for compression workbench as well as runtime improvements. Implementation fo a HashNode which tries to increase the runtime, is however slower as the leaders node. Further the adoption of the hierarchy import manager to account for brushing was attempted, some error still exists. | 16 February 2022, 18:00:05 UTC |
87cc287 | Josef Stumpfegger | 14 February 2022, 13:42:12 UTC | Fixed small bug in hierarchy creation, implemented adoption to drawlist to be able to use the hierarchy manager reloading to properly show the dataset. | 14 February 2022, 13:42:12 UTC |
dd4b3de | Josef Stumpfegger | 13 February 2022, 18:07:40 UTC | Changed drawlist creation to be able to handle hierarchical files. Hierarchy building seems to have some problem(combination of compressed files). | 13 February 2022, 18:07:40 UTC |
7ffdb9f | Josef Stumpfegger | 13 February 2022, 07:45:21 UTC | Compilable version. Continuing to adopt create drawlist. | 13 February 2022, 07:45:21 UTC |
542811b | Josef Stumpfegger | 12 February 2022, 19:15:41 UTC | Opening of hierarchy in the PCViewer was added. Creation and drawing of hierarchical drawlists with automatic reloading not yet implemented. | 12 February 2022, 19:15:41 UTC |
e1371e1 | Josef Stumpfegger | 11 February 2022, 18:22:07 UTC | Started adoptions for main pcp rendering to be able to also include changin drawlist data. | 11 February 2022, 18:22:07 UTC |
0c849c1 | Josef Stumpfegger | 11 February 2022, 18:01:49 UTC | bugfix for inBrush function. | 11 February 2022, 18:01:49 UTC |
12119bc | Josef Stumpfegger | 11 February 2022, 18:01:27 UTC | First iplementation for the Hierarchy import manger to automatically determine the hierarchy levels done. Compilable, but not yet tested. | 11 February 2022, 18:01:27 UTC |
2038d61 | Josef Stumpfegger | 09 February 2022, 20:06:01 UTC | Rtree in leaders clustering is now static for up to 30 dimensions, above rtree has nodes with dynamic size dimension arrays. Not yet fully tested and debugged. | 09 February 2022, 20:06:01 UTC |
4eb5e5a | Josef Stumpfegger | 09 February 2022, 18:15:29 UTC | Added rtree util with the variant. | 09 February 2022, 18:15:29 UTC |
7462a19 | Josef Stumpfegger | 09 February 2022, 18:15:04 UTC | Changing rtree to a rtree variant which adopts to the amount of dimensions (uses a static version of rtree which is faster). | 09 February 2022, 18:15:04 UTC |
0e567c8 | Josef Stumpfegger | 08 February 2022, 13:22:29 UTC | Compile fixes under ubuntu. | 08 February 2022, 13:22:29 UTC |
91fc68a | Josef Stumpfegger | 07 February 2022, 17:31:44 UTC | Added documentation for hierarchical file layouts and compression methods. | 07 February 2022, 17:31:44 UTC |
d9efd8b | Josef Stumpfegger | 04 February 2022, 17:54:54 UTC | Started import manager implementation with automatic adoption to brushes. | 04 February 2022, 17:54:54 UTC |
7ec073a | Josef Stumpfegger | 03 February 2022, 18:31:13 UTC | Implemented merging of temporary clusters with following compression via dwt, quantization, run length encoding and last huffman encoding. Correct writeout to files as well as correct import of compressed file. The compression also automatically converts rwo major points to column major to be able to faster convert to data object and to have more efficient compression. | 03 February 2022, 18:31:13 UTC |
95c6a8a | Josef Stumpfegger | 02 February 2022, 20:30:31 UTC | Dwt as well as quantization are working, cpu compression still has some errors. Main contains debug code. | 02 February 2022, 20:30:31 UTC |
ce6f95e | Josef Stumpfegger | 02 February 2022, 08:10:17 UTC | Added abstract class for clustering nodes. | 02 February 2022, 08:10:17 UTC |
2a3a6a3 | Josef Stumpfegger | 29 January 2022, 18:48:16 UTC | Added cudaCompress for cpu to encode the cache files. Rework was done so that only the cpu version of cuda compress is available and is compilable when all files are inthe same file. | 29 January 2022, 18:48:16 UTC |
a3456af | Josef Stumpfegger | 29 January 2022, 18:10:49 UTC | Deadlock fix, more info in the gui, file writeout fixed, multhtread implementation nearly completly done (Missing proper canelation fo hierarchy building). | 29 January 2022, 18:10:49 UTC |
241a549 | Josef Stumpfegger | 28 January 2022, 18:18:58 UTC | Added caching if ram is being filled up. Hierarchy creation is now able to run multithreaded. Thread amount is settable, insertions are lock free and secure. Writing out the chaceh files has still some errors. | 28 January 2022, 18:18:58 UTC |
ff3554d | Josef Stumpfegger | 27 January 2022, 19:20:49 UTC | bugfixes for hierarchy creation. | 27 January 2022, 19:20:49 UTC |
ee6e6a8 | Josef Stumpfegger | 27 January 2022, 10:16:15 UTC | Implemented Hierarchy Node to be able to recursiveley add datapoints. First take on tmp hirarchy creation implemented using rtrees as acceleration structures. Current version compilable without errors and warnings. | 27 January 2022, 10:16:15 UTC |
e2b177c | Josef Stumpfegger | 25 January 2022, 15:10:02 UTC | Compression workbench is now usable from within the pcviewer application. Data anlysis is moved to a separate thread. | 25 January 2022, 15:10:02 UTC |
6388e04 | Josef Stumpfegger | 25 January 2022, 13:56:30 UTC | Fixed small bugs in loader, implemented new compression workbench gui to do data loading in the PCViewer. | 25 January 2022, 13:56:30 UTC |
771e1c3 | Josef Stumpfegger | 21 January 2022, 19:20:52 UTC | Started implementatio of Hirarchy building. NetCdf loader nearly done. | 21 January 2022, 19:20:52 UTC |
c685d1b | Josef Stumpfegger | 18 January 2022, 17:14:31 UTC | Tested framebuffer addition with a single component float framebuffer-> same render times as normal rendering if opacity = 1/255, else slower(but constantly slow). | 18 January 2022, 17:14:31 UTC |
1c5ddb0 | Josef Stumpfegger | 18 January 2022, 15:00:43 UTC | Working implementation for alternative parallel coordinates rendering via atomic add on image with a resolve pass. Unfortunately around 10 times slower. Improvement: use float buffer to be able to use output merger again. | 18 January 2022, 15:00:43 UTC |
c16a68e | Josef Stumpfegger | 17 January 2022, 18:16:52 UTC | Compilable and runnable version. Does not yet work as the image has to be added to the fragment shader stage via descriptor set to be able to atomic add into the image. | 17 January 2022, 18:16:52 UTC |
cf6ff49 | Josef Stumpfegger | 17 January 2022, 17:04:31 UTC | First done implementation. Not yet checked for compilation and bugs. | 17 January 2022, 17:04:31 UTC |
16b4e66 | Josef Stumpfegger | 17 January 2022, 14:01:15 UTC | Inbetween commit for efficient rendering pipeline. | 17 January 2022, 14:01:15 UTC |
31c9595 | Josef Stumpfegger | 07 January 2022, 16:50:08 UTC | Further implemented dataset gpu sorting. | 07 January 2022, 16:50:08 UTC |
255f3f8 | Josef Stumpfegger | 05 January 2022, 10:01:14 UTC | Added data access to dataset index sorting. Sorting still has to be adopted to use the dataset. | 05 January 2022, 10:01:14 UTC |
115e137 | Josef Stumpfegger | 03 January 2022, 15:06:05 UTC | Hybrid radix sort improvement by better loading strategy. | 03 January 2022, 15:06:05 UTC |
861e621 | Josef Stumpfegger | 29 December 2021, 09:52:18 UTC | Gpu radix sorter improvement, started to add things for dataset sorting. | 29 December 2021, 09:52:18 UTC |