| 92a95e3 | rmar3a | 17 November 2020, 18:41:17 UTC | Moved the runtime code for the buildPVarsCounts() method. - Moved the runtime code for the buildPVarsCounts() method into the method to prevent code duplication of the runtime code. | 18 November 2020, 02:18:16 UTC |
| f70fc93 | rmar3a | 17 November 2020, 02:12:32 UTC | Local counts tables are no longer generated for the hybrid method for PVars. - Local counts tables are no longer generated for the hybrid method for PVars. Instead the global counts tables are read from during the Moebius Join. | 18 November 2020, 02:18:15 UTC |
| 9f40f1d | oschulte | 24 September 2020, 17:19:42 UTC | Merge pull request #222 from sfu-cl-lab/improved-bifchecker Updated the bifchecker script to support the XML file in the res directory. | 24 September 2020, 17:19:42 UTC |
| f03cd6b | rmar3a | 24 September 2020, 00:59:35 UTC | Updated the bifchecker script to support the XML file in the res directory. - In the case when there are issues with generating the BIF file via FactorBase, the bifchecker.py script has been updated to support extracting metrics from the XML file that is produced in the "res" directory created by FactorBase. | 24 September 2020, 01:01:51 UTC |
| e4bdc8b | oschulte | 16 September 2020, 22:07:14 UTC | Merge pull request #221 from sfu-cl-lab/improved-metadata Improved metadata | 16 September 2020, 22:07:14 UTC |
| 9b1b137 | rmar3a | 11 September 2020, 21:48:50 UTC | Replaced the local lattice_mapping table with a view. - Replaced the local lattice_mapping table with a view that retrieves all the mappings instead of pre-filtering out the results. | 11 September 2020, 23:19:02 UTC |
| cb8dfd7 | rmar3a | 11 September 2020, 21:05:49 UTC | Changed local FNodes table into a view. - The local FNodes table is only referenced once in the code so it should be more efficient to use a view instead of a table. - Have to convert the FNodes view into a table so that UPDATE calls in CP.java work. | 11 September 2020, 23:18:56 UTC |
| 6bd3099 | rmar3a | 11 September 2020, 09:35:19 UTC | Removed unnecessary table JOIN in the metaqueries. - When adding some entries to the MetaQueries table we JOINed the RNodes_2Nodes with the 2Nodes table to get the 2Nodes associated with the RNodes of the current family, but this isn't necessary as RNodes_2Nodes already only contains the 2Nodes associated with the RNodes of the current family. | 11 September 2020, 09:35:19 UTC |
| e8b6298 | oschulte | 22 July 2020, 17:59:31 UTC | Merge pull request #220 from sfu-cl-lab/console-output "_counts" and "_CT" Table Runtimes Written to the Console Output. | 22 July 2020, 17:59:31 UTC |
| e60f069 | rmar3a | 19 July 2020, 23:35:38 UTC | Improved the logic for creating the "_counts" tables for RChains. - Improved the logic for determining when/if the "_counts" tables for RChains should be created. - When building the "_counts" tables for the global counts or precount we should start from an RChain length of 1 since we need to build all the tables. - When building the "_counts" tables for ondemand or hybrid we should start from an RChain length of 2 since we cache the CT tables meaning that we might not need to construct the "_counts" table for some RNodes. Also, for hybrid we use subqueries instead of materializing the tables for RNodes, so creating the tables is unnecessary for RNodes. - Updated the buildRChainCounts() method to take in a starting height since it's easier to determine this value outside the method. | 19 July 2020, 23:45:05 UTC |
| e87d2fc | rmar3a | 19 July 2020, 13:21:33 UTC | Executed queries are only displayed when logging set to debug level. - Most of the SQL queries that are executed are only written to the console when the log level is set to "debug" now. - Created a new method logExecutedQuery() to help write the executed SQL query out to the console in a consistent format. | 19 July 2020, 14:02:03 UTC |
| 4ac1e62 | rmar3a | 19 July 2020, 13:12:03 UTC | Removed additional unnecessary logging statements. - Removed logging statements that printed unnecessary runtimes. - Removed logging statements that printed information that wasn't too helpful. - Removed logging statements that printed duplicate information. | 19 July 2020, 14:02:03 UTC |
| 896e452 | rmar3a | 19 July 2020, 05:07:36 UTC | Console output logging for "_counts" and "_CT" table creation. - Added a new logging level, "runtimeDetails," to enable console logging of the runtime for creating the "_counts" and "_CT" tables. - Created the method logRunTimeDetails() to help write the console output of the runtimes consistently. - Also output the build time for the "_counts" tables for PVars so that the times add up to the value found on the line that looks like: Building Time(ms) for ALL CT tables: 448 ms. | 19 July 2020, 14:01:59 UTC |
| 4386785 | rmar3a | 15 July 2020, 22:50:44 UTC | Removed more unnecessary logging statements. - Removed logging statements that didn't provide much information. | 15 July 2020, 22:50:44 UTC |
| 9416326 | rmar3a | 15 July 2020, 21:08:11 UTC | Removed unnecessary logging statements. - There were logging statements that printed out components of queries to the console; however, this shouldn't be necessary as we can just print out the final query which contains all the components. | 15 July 2020, 21:08:11 UTC |
| 4ad0b2c | rmar3a | 15 July 2020, 19:08:02 UTC | Removed duplicate code for printing the Structure Learning time. - Removed the duplicate code for printing the Structure Learning time. | 15 July 2020, 19:08:02 UTC |
| c5b75fe | oschulte | 14 July 2020, 23:41:32 UTC | Merge pull request #219 from sfu-cl-lab/ondemand-improvement Ondemand improvement | 14 July 2020, 23:41:32 UTC |
| 018766a | oschulte | 14 July 2020, 23:41:12 UTC | Merge pull request #217 from sfu-cl-lab/large-database-support Changes required for large DBs like VisualGenome. | 14 July 2020, 23:41:12 UTC |
| e0c4ad0 | rmar3a | 14 July 2020, 01:53:26 UTC | Updated output. - The output includes the input database name. - Adjusted the casing for the counting strategy to be consistent with the rest of the output. | 14 July 2020, 01:53:26 UTC |
| 508d5d6 | rmar3a | 14 July 2020, 01:47:21 UTC | Counts tables for RNodes are created when using Ondemand counting. - For Ondemand counting, the counts tables for RNodes are created once and read from instead of using subqueries like the hybrid method to avoid executing expensive joins multiple times. - Created a new method generateCountsTable() which is used to generate the "_counts" tables for RNodes. - Corrected/added more Javadoc. | 14 July 2020, 01:51:02 UTC |
| f9c1062 | rmar3a | 14 July 2020, 01:21:34 UTC | Removed unnecessary variable. - The end time for constructing an RNode counts table isn't necessary as the time can be computed inline so the variable has been removed. | 14 July 2020, 01:21:34 UTC |
| cdc0834 | oschulte | 10 July 2020, 17:27:32 UTC | Merge pull request #216 from sfu-cl-lab/structure-learning-only Structure Learning Only Option | 10 July 2020, 17:27:32 UTC |
| bc7bfbb | oschulte | 10 July 2020, 17:23:15 UTC | Merge pull request #215 from sfu-cl-lab/no-false-star Subqueries for "_false" and "_star" tables. | 10 July 2020, 17:23:15 UTC |
| 50e8f5a | oschulte | 08 July 2020, 16:52:08 UTC | Merge pull request #218 from sfu-cl-lab/ondemand-correction Disabled generation of global counts for the Ondemand strategy. | 08 July 2020, 16:52:08 UTC |
| 3224354 | rmar3a | 08 July 2020, 15:47:31 UTC | Disabled generation of global counts for the Ondemand strategy. - The global counts is only for the Hybrid method so it has been disabled for the Ondemand strategy. | 08 July 2020, 15:47:31 UTC |
| 72157c8 | rmar3a | 17 April 2020, 19:08:28 UTC | Changes required for large DBs like VisualGenome. - Changed the datatype for numbers from Long to Double on the Java side to be able to support larger numbers. - Increased the length of the VARCHAR for RNode related columns, which could sometimes truncate if the RChain names got too long. - Changed the datatype for numbers from BIGINT to DECIMAL on the MySQL side to be able to support larger numbers. | 06 July 2020, 19:21:17 UTC |
| 51eb905 | rmar3a | 04 July 2020, 09:49:12 UTC | [deadcode] Removed unnecessary short_rnid to orig_rnid mapping code. - Removed unnecessary short_rnid to orig_rnid mapping code from the Final_Path_BayesNets() method since the mapping is no longer necessary as the short_rnid is only used for the table names and not used in the other tables anymore. | 04 July 2020, 09:52:02 UTC |
| a1de0e9 | rmar3a | 04 July 2020, 09:44:57 UTC | Added an option to skip parameter learning. - Added an option to skip parameter learning. - Moved the call to the exportResults() method to earlier so that the XML file is built regardless of whether or not parameter learning is done. | 04 July 2020, 09:44:57 UTC |
| b8b817b | rmar3a | 30 June 2020, 09:13:15 UTC | No more physical creation of the "_counts" tables for RNodes. - Instead of creating the "_counts" tables on disk or memory for RNodes, an equivalent subquery is used to help generate the "_counts" tables for RNodes instead. - Adjusted the names of some variables to better reflect their purpose. Note: The "_counts" tables for the global counts are still created physically. | 30 June 2020, 22:38:11 UTC |
| 31edf0a | rmar3a | 28 June 2020, 03:47:43 UTC | No more physical creation of the "_star" tables. - Instead of creating the "_star" tables on disk or memory, an equivalent subquery is used to help generate the "_star" tables instead. - Updated the createSubtractionQuery() and sort_merge() methods to handle a subquery instead of a table name. - Updated the associated tests and Javadoc. - Updated the components runners module to properly use this new code. - Adjusted the names of some variables to better reflect their purpose. | 28 June 2020, 03:49:28 UTC |
| 91548dd | rmar3a | 27 June 2020, 10:24:25 UTC | Updated the components runner module to support Connector/J 8.0.19. - Similar to this commit: 94f312daf3ebce5c09c8f07e1ca5625f5c2c68cc - Specified the serverTimezone parameter in the connection string so that an invalid serverTimezone value wouldn't be used and cause the program to crash. | 28 June 2020, 03:49:28 UTC |
| f5bfcab | rmar3a | 27 June 2020, 09:37:27 UTC | No more physical creation of the "_false" tables. - Instead of creating the "_false" tables on disk or memory, an equivalent subquery is used to help generate the "_false" tables instead. - Updated the associated tests and Javadoc. - Updated the components runners module to properly use this new code. | 28 June 2020, 03:49:23 UTC |
| 366379a | rmar3a | 26 June 2020, 23:31:03 UTC | Added new buildRNodeCounts column to the CallLogs table. - Instead of making 2 database calls to update the runtimes for the "Counts" and "Moebius Join" portions, a single database call is made to update the buildRNodeCounts column. - The runtime from the buildRNodeCounts column is added to the "Counts" portion and subtracted from the "Moebius Join" portion to get a more correct runtime. | 27 June 2020, 08:28:49 UTC |
| 92b0876 | oschulte | 25 June 2020, 00:25:15 UTC | Merge pull request #213 from sfu-cl-lab/database-ct-cache Database CT Cache Plus a Couple Other Changes | 25 June 2020, 00:25:15 UTC |
| 3041145 | rmar3a | 24 June 2020, 23:17:21 UTC | _CT_cache database feature is disabled for Precount counting strategy. - Updated the code pass in a CountStrategy enum to help avoid using the _CT_cache database for the Precount counting strategy. - Updated the Javadoc to reflect this change. | 24 June 2020, 23:21:39 UTC |
| e02d87f | rmar3a | 24 June 2020, 10:14:10 UTC | Enable _CT_cache database feature for Ondemand counting strategy. - The Ondemand counting strategy now uses the _CT_cache database when generating counts tables for PVars and CT tables for RNodes, which is the same as the Hybrid counting strategy. - Updated CountingStrategy enum to have a useCTCache() method used to determine if the _CT_cache database should be used. - Updated methods to use a CountingStrategy enum instead of multiple variables where possible. - Updated the Javadoc to reflect this change. | 24 June 2020, 23:18:53 UTC |
| 077b4cb | oschulte | 24 June 2020, 22:06:06 UTC | Merge pull request #214 from sfu-cl-lab/faster-metaqueries Moved the view in metaqueries_RChain.sql to metaqueries_initialize.sql. | 24 June 2020, 22:06:06 UTC |
| 34b8758 | rmar3a | 23 June 2020, 22:24:51 UTC | Generating counts for RNodes takes advantage of the _CT_cache database. - The "_counts" tables are no longer generated for RNodes if an existing "_CT" table exists in the "_CT_cache" database. - Moved the buildRChainCounts() method into the CTGenerator() method to group similar methods together. - Updated the buildRNodeCTCreationQuery() method to get column information from the MetaQueries table instead of from a physically created table. - Adjusted the runtime values recorded in the CallLogs table for the counts and Moebius Join portions otherwise the runtime for the counts portion was being added to the Moebius Join portion. - Changed the CallLogs table to use the MEMORY engine so that the time adjustment calls for the counts and Moebius join don't affect the runtime too much. | 23 June 2020, 23:19:23 UTC |
| bb61921 | rmar3a | 23 June 2020, 21:32:41 UTC | Database logging method for adjusting an existing value in the CallLogs table. - Created a database logging method for adding to/subtracting from an existing value in the CallLogs table. | 23 June 2020, 21:34:52 UTC |
| 81e4390 | rmar3a | 19 June 2020, 03:51:18 UTC | Cleaned up the if/else block in the buildRChainCounts() method. - Cleaned up the if/else block in the buildRChainCounts() method by extracting the single different parameter of the same method being called into a variable. | 19 June 2020, 03:51:18 UTC |
| 0ae059d | rmar3a | 19 June 2020, 01:56:41 UTC | Avoid calling the method buildRNodeCTCreationQuery() twice. - The method buildRNodeCTCreationQuery() was called in the methods buildRNodesCT() and buildRNodesCTFromCache() where the method buildRNodesCTFromCache() calls buildRNodesCT() if the table doesn't exist in the cache yet. This caused the method buildRNodeCTCreationQuery() to be called twice unnecessarily. - Moved the call to buildRNodeCTCreationQuery() outside the two mentioned above and pass the String generated by the buildRNodeCTCreationQuery() method into the two methods. - Updated the Javadoc to reflect this change. | 19 June 2020, 02:18:04 UTC |
| 2fc1909 | rmar3a | 18 June 2020, 22:15:12 UTC | Moved the view in metaqueries_RChain.sql to metaqueries_initialize.sql. - Moved the view in metaqueries_RChain.sql to metaqueries_initialize.sql since there is no point regenerating the view multiple times. - This change makes generation of the MetaQueries table faster. | 18 June 2020, 22:15:12 UTC |
| b1b4e32 | rmar3a | 18 June 2020, 11:45:30 UTC | CallLogs records JOIN table runtime. - The CallLogs table now records the runtime for generating the derived JOIN table String that is created by the createJoinTableQueries() method. - This runtime should make a negligible contribution to the overall runtime unless the implementation of createJoinTableQueries() changes and so is just included for completion. | 18 June 2020, 12:08:18 UTC |
| 83a2f14 | rmar3a | 18 June 2020, 11:30:40 UTC | Created a method to help generate the CREATE query strings for views. - Created a method to help generate the CREATE query strings for and used it as much as possible in CountsManager.java and Sort_merge3.java. - This will hopefully help reduce the number of implicit StringBuilder objects that are created for String concatenation. Note: The method wasn't used for some queries since using the method would not remove the implicit creation of an extra StringBuilder object. | 18 June 2020, 12:08:11 UTC |
| 8de0a18 | rmar3a | 18 June 2020, 10:57:56 UTC | Created a method to help generate the CREATE query strings for tables. - Created a method to help generate the CREATE query strings for tables and used it as much as possible in CountsManager.java. - This will hopefully help reduce the number of implicit StringBuilder objects that are created for String concatenation. Note: The method wasn't used for some queries since using the method would not remove the implicit creation of an extra StringBuilder object. | 18 June 2020, 11:32:54 UTC |
| bf83c74 | rmar3a | 17 June 2020, 21:56:57 UTC | Added a CT cache for RNodes. - It was observed that we were regenerating the exact same "_CT" tables multiple times for the RNodes since the "_CT" database is deleted each time the buildCT() method is called. - The "_CT" tables for RNodes in the "_CT" database are now views that point to the correct "_CT" table in the CT cache database. - Created a new @FunctionalInterface interface to help switch between generating the "_CT" tables by checking the cache first or just generating them from scratch always. - Updated the buildRNodesCT() method to process a single RNode at a time so that it could be used by the new buildRNodesCTFromCache() method. - Moved the start time for generating the CT tables a few lines higher to more accurately capture this part of the process, although the change in time should be negligible. - Renamed the method buildRNodeCTCreateQuery() to buildRNodeCTCreationQuery() since the method now returns the portion of the CREATE query after the "AS" keyword. | 18 June 2020, 08:45:34 UTC |
| c08ef28 | rmar3a | 15 June 2020, 10:38:41 UTC | Renamed variables related to CT cache. - Renamed additional variables I forgot to rename in this commit: 3f907bab8d5828af9d931e583a8086fd47904ade | 17 June 2020, 10:21:48 UTC |
| c371cbf | oschulte | 16 June 2020, 17:01:00 UTC | Merge pull request #209 from sfu-cl-lab/code-cleanup Various Code Cleanup | 16 June 2020, 17:01:00 UTC |
| 8e9ea8e | rmar3a | 15 June 2020, 21:32:59 UTC | Updated the buildRNodeCT() method to return a String. - Updated the buildRNodeCT() method to return a String instead of building a table. - Renamed buildRNodeCT() to buildRNodeCTCreateQuery() to reflect this change. | 15 June 2020, 21:32:59 UTC |
| 3f907ba | rmar3a | 15 June 2020, 10:38:41 UTC | Renamed the "*_counts_cache" database to "*_CT_cache". - Renamed the "*_counts_cache" database to "*_CT_cache" to better reflect the fact that the tables stored in the "*_CT_cache" database are the ones found in the "*_CT" database. - Updated the associated variable name of the HashMap used for storing the information for the cached tables. | 15 June 2020, 11:32:48 UTC |
| ecae622 | rmar3a | 15 June 2020, 10:20:08 UTC | Moved the RNode ID extraction code from buildRNode* methods to buildRNodesCT. - Moved the RNode ID extraction code from the buildRNode* methods to the buildRNodesCT method. - Updated the buildRNode* methods to take in the name/short name of the RNodes to build the CT tables for instead of a FunctorNodesInfo object. - Updated the Javadoc and a comment. | 15 June 2020, 10:34:46 UTC |
| 60e1fc9 | rmar3a | 13 June 2020, 12:03:05 UTC | Removed unnecessary DISTINCT keyword from a query in CountsManager.java. - Removed a DISTINCT keyword from a query in CountsManager.java since it isn't necessary. | 13 June 2020, 18:56:32 UTC |
| 36b7f6f | rmar3a | 13 June 2020, 08:45:19 UTC | Made the MetaQueries queries more consistent. - Adjusted the casing for queries related to the MetaQueries table so that the TableType field is always capitialized only and so that the ClauseType field is always all uppercase. - Updated mysql-extraction.txt to reflect these changes. | 13 June 2020, 18:56:10 UTC |
| fd199c3 | rmar3a | 13 June 2020, 11:31:51 UTC | Updated the createMetaQueriesExtractionQuery() method to support EntryType. - Updated the createMetaQueriesExtractionQuery() method to support the EntryType column of the MetaQueries table and used this feature for some queries. - Updated existing uses of the method. | 13 June 2020, 11:49:41 UTC |
| 1ec4d2e | rmar3a | 11 June 2020, 01:23:09 UTC | Moved many String concatenations for MetaQueries into a method. - There were many String concatenations done by extracting "Entries" from the MetaQueries table that looked very similar so this query is now generated by the createMetaQueriesExtractionQuery() method in QueryGenerator.java. | 13 June 2020, 10:51:58 UTC |
| a0d074c | rmar3a | 10 June 2020, 22:51:15 UTC | Removed unnecessary if/else blocks in BayesBaseH.java. - There were two if/else blocks in BayesBaseH.java that weren't necessary as the value of the variable being assigned in the if/else blocks could be assigned directly from the result of the code in the condition of the if statement so they have been removed. | 10 June 2020, 22:52:54 UTC |
| f07ab7b | rmar3a | 10 June 2020, 20:52:35 UTC | Use a single instance of StringBuilder for QueryGenerator. - Use a single instance of StringBuilder for QueryGenerator so that we don't have to recreate the object. - Replaced the use of ArrayList with StringJoiner since the purpose of ArrayList and String.join() can be replaced by StringJoiner. - Replaced all String "+" operations with equivalent StringBuilder expressions to avoid having extra unnecessary StringBuilders being added by the compiler. | 10 June 2020, 22:45:26 UTC |
| 61c2219 | rmar3a | 10 June 2020, 22:41:00 UTC | Deleted the makeDelimitedString() method. - Deleted the makeDelimitedString(List<String> columns, String delimiter) method since the logic it contained is no longer useful. - Uses of the makeDelimitedString() method were replaced with equivalent String.join() calls. | 10 June 2020, 22:41:00 UTC |
| 44df2ee | oschulte | 10 June 2020, 21:52:11 UTC | Merge pull request #212 from sfu-cl-lab/ct-cleanup Changed the build order for the star, flat, false, and CT tables. | 10 June 2020, 21:52:11 UTC |
| be326f0 | rmar3a | 10 June 2020, 10:56:16 UTC | Moved "_false" table logic into a separate method. - Moved "_false" table logic from the buildRNodeCT method into a separate method to make it more clear how the methods relate to the usage of the sort merge algorithm. - Cleaned up the code and Javadoc. | 10 June 2020, 18:19:29 UTC |
| c4afed2 | rmar3a | 10 June 2020, 10:35:35 UTC | [build-order] (3/3) Updated the buildRNodeFlat() method. - Updated the buildRNodeFlat() method to process one RNode instead of many. - Cleaned up the code. | 10 June 2020, 18:19:25 UTC |
| 987d204 | rmar3a | 10 June 2020, 10:00:55 UTC | [build-order] (2/3) Updated the buildRNodeStar() method. - Updated the buildRNodeStar() method to process one RNode instead of many. - Adjusted the order of the methods buildRNodeStar and buildRNodeFlat so that they better match how they are used in the sort merge part of the algorithm. - Cleaned up the code. | 10 June 2020, 17:45:37 UTC |
| 6a05ca0 | rmar3a | 09 June 2020, 22:12:41 UTC | [build-order] (1/3) Build the CT tables one at a time for each RNode. - Build the CT tables one at a time instead of by stage. Prior to this change, we built all the "_star", then all the "_flat", and then all the "_false" and "_CT" tables. After this change we build the required "_star", "_flat", and "_false" tables for a "_CT" table, and then build the "_CT" table (this is done for each CT table). - Renamed the following methods: BuildCT_Rnodes_flat --> buildRNodeFlat BuildCT_Rnodes_star --> buildRNodeStar BuildCT_Rnodes_CT --> buildRNodeCT - Updated the buildRNodeCT() method to process one RNode instead of many. - Cleaned up the code. | 10 June 2020, 17:45:37 UTC |
| 0e79917 | oschulte | 09 June 2020, 23:50:34 UTC | Merge pull request #211 from sfu-cl-lab/ct-cleanup Cleaned up some of the CT code. | 09 June 2020, 23:50:34 UTC |
| 229a7a6 | rmar3a | 09 June 2020, 10:30:01 UTC | Removed the "parent" column from a SELECT query in buildRChainsCT(). - The "parent" column was included for a SELECT query in the buildRChainsCT() method, but wasn't used for anything useful so it has been removed from the query. - Also removed the DISTINCT keyword since it's not necessary. | 09 June 2020, 20:25:59 UTC |
| ac93606 | rmar3a | 09 June 2020, 09:57:27 UTC | Moved the BuildCT_Rnodes_* methods into a separate method. - Moved the BuildCT_Rnodes_* methods into a separate method called buildRNodesCT() to help organize the code. - Renamed the BuildCT_RChain_flat() method to buildRChainsCT() to better match the new buildRNodesCT() method. - Moved a large block of comments from the CTGenerator() method to the buildRChainsCT() method since they seemed to make more sense for the buildRChainsCT() method. - Made some minor adjustments to some of the comments. - Removed some unnecessary comments and logger statements. - Cleaned up some white-space. | 09 June 2020, 20:25:59 UTC |
| 4b8b54d | rmar3a | 06 June 2020, 01:19:46 UTC | Removed some unnecessary white-space. | 06 June 2020, 09:11:37 UTC |
| 9d479c8 | rmar3a | 06 June 2020, 09:06:05 UTC | Drop_tmpTables() method can drop views in addition to tables. - Cleaned up the Drop_tmpTables() method so that it is more similar to the rest of the files in the codebase. - Updated the Drop_tmpTables() method to be able to drop views in addition to tables. | 06 June 2020, 09:09:24 UTC |
| eb268b6 | rmar3a | 06 June 2020, 01:16:48 UTC | Changed the "_false" tables to views. - Changed the "_false" tables to views since these are just intermediate tables in the current implementation for the hybrid counting strategy. - Removed the storageEngine parameter since we use views instead and updated any references to the sort_merge method. - Removed some duplicated code. - Updated the test code for the sort_merge method. | 06 June 2020, 07:10:10 UTC |
| 2c884db | rmar3a | 06 June 2020, 00:59:09 UTC | Changed the "_star" tables to views. - Changed the "_star" tables to views since these are just intermediate tables in the current implementation for the hybrid counting strategy. | 06 June 2020, 01:00:27 UTC |
| fd2b708 | oschulte | 05 June 2020, 16:42:58 UTC | Merge pull request #208 from sfu-cl-lab/counts-tables-cache Added a counts cache for population variables. | 05 June 2020, 16:42:58 UTC |
| 413f79c | rmar3a | 04 June 2020, 23:55:42 UTC | Added a counts cache for population variables. - It was observed that we were regenerating the exact same "_counts" tables multiple times for the population variables since the "_CT" database is deleted each time the buildCT() method is called. - Created a new "_counts_cache" database used to store the various combinations of columns used when generating the "_counts" tables for population variables. - The "_counts" tables for population variables in the "_CT" database are now views that point to the correct "_counts" table in the counts cache database. - Created a new @FunctionalInterface interface to help switch between generating the "_counts" tables by checking the cache first or just generating them from scratch always. - Cleaned up some of the code. | 04 June 2020, 23:55:42 UTC |
| e1fc4c5 | oschulte | 03 June 2020, 16:39:48 UTC | Merge pull request #207 from sfu-cl-lab/minor-adjustments Minor adjustments | 03 June 2020, 16:39:48 UTC |
| e537a13 | rmar3a | 29 May 2020, 20:36:25 UTC | Updated a couple queries to use an executable comment. - Setting the collation to utf8_general_ci was specific to our new database servers which use MariaDB 10.3.16, so the collation changing are now placed into executable comments so that they are only executed when run on specific versions of MariaDB. | 03 June 2020, 01:00:28 UTC |
| aa21ef3 | rmar3a | 29 May 2020, 20:32:49 UTC | Cleaned up a couple lines of code in CountsManager.java. - The variable fromAliases is only used within an "if" statement so declaration of this variable has been moved into the "if" statement. | 03 June 2020, 00:40:42 UTC |
| 4f94020 | oschulte | 02 June 2020, 22:43:52 UTC | Merge pull request #206 from sfu-cl-lab/memory-toggled MEMORY storage engine toggled based on the counting strategy. | 02 June 2020, 22:43:52 UTC |
| ad1eb8a | rmar3a | 31 May 2020, 01:59:38 UTC | Automatically set storage engine for tables based on counting strategy. - Added a new getStorageEngine() method to the CountingStrategy enum that can be used to set the storage engine for the "_star", "_flat", "_false", "_CT", and "_counts" based on the counting strategy. - Replaced the hard coded MEMORY storage engine settings with a variable. - Updated the parameters for several methods to take in a String parameter to specify the storage engine that should be used for the tables that the method generates. - Updated the calls to these methods to reflect these changes. - Updated some of the Javadoc. | 01 June 2020, 20:40:43 UTC |
| 0e886d1 | rmar3a | 29 May 2020, 22:54:06 UTC | Cleaned up the sort merge code. - Adjusted the order of the parameters for the sort_merge method so that it is more similar to the other methods in the codebase. - Updated calls to sort_merge to reflect the change in the parameter order. - Added Javadoc to the sort_merge method. - Changed the name of the table3 variable to outputTableName to help clarify the purpose of the variable. - Removed the unnecessary "Statement" casting. - Removed the unnecessary "time5" variable. - Removed an unnecessary import in Sort_merge3Test.java. - Changed the "CREATE + INSERT" method of creating the outputTableName table to the "CREATE AS" method and updated the sort_merge method to alias the subtraction of the columns to their shared column name. | 01 June 2020, 20:40:31 UTC |
| 613fbc2 | oschulte | 29 May 2020, 21:29:17 UTC | Merge pull request #205 from sfu-cl-lab/remove-columnstore-code Removed the ColumnStore code. | 29 May 2020, 21:29:17 UTC |
| 1716d22 | oschulte | 29 May 2020, 21:27:08 UTC | Merge pull request #204 from sfu-cl-lab/global-counts-logging Updated the runtime logging for the hybrid method. | 29 May 2020, 21:27:08 UTC |
| b09b1ec | oschulte | 29 May 2020, 21:23:27 UTC | Merge pull request #203 from sfu-cl-lab/updated-tests Updated the test database code to support Connector/J 8.0.19. | 29 May 2020, 21:23:27 UTC |
| ec231ec | rmar3a | 29 May 2020, 20:24:22 UTC | Removed the ColumnStore code. - Removed the ColumnStore specific code introduced in this commit: a7361f0637323b3b1046d0e44c924a4823f09d3a - We attempted to use the ColumnStore engine, but it didn't seem to perform faster overall. Although running simple projection queries on the ColumnStore tables seemed a lot faster compared to MEMORY tables! - Made backup copies of the files in the obsolete directory so we can view this code more easily if we want to explore ColumnStore engine tables again in the future. | 29 May 2020, 20:30:52 UTC |
| c8a1fc3 | rmar3a | 29 May 2020, 03:52:31 UTC | Updated the runtime logging for the hybrid method. - The CallLogs table now records the runtime for generating the global counts database. | 29 May 2020, 03:56:04 UTC |
| 318fa1a | rmar3a | 28 May 2020, 04:02:28 UTC | Updated the test database code to support Connector/J 8.0.19. - Updated the test code used to connect to the tests database so that it sets the necessary connection properties that might be incompatible with Connector/J 8.0.19. | 28 May 2020, 04:03:45 UTC |
| 8186334 | oschulte | 15 May 2020, 22:41:27 UTC | Merge pull request #202 from sfu-cl-lab/corrections Adjusted value for MAX_NUM_OF_PVARS and updated a query. | 15 May 2020, 22:41:27 UTC |
| 738eeab | rmar3a | 15 May 2020, 06:39:24 UTC | Reduced the value of MAX_NUM_OF_PVARS from 10 to 5. - When generating the relationship lattice, valid RChains are restricted to ones that have a maximum of 5 population variables instead of 10, which is a more reasonable value. | 15 May 2020, 19:34:26 UTC |
| 21883cb | rmar3a | 15 May 2020, 06:25:09 UTC | Corrected the query in the learnStructurePVars() method. - The EntityTables table is no longer copied from the "_setup" database so the SELECT query in the learnStructurePVars() method needed to be updated to read the table from the "_setup" database. | 15 May 2020, 19:34:25 UTC |
| 9f9a291 | oschulte | 27 March 2020, 21:41:41 UTC | Merge pull request #201 from sfu-cl-lab/db-logging Added database logging to RuntimeLogger. | 27 March 2020, 21:41:41 UTC |
| 0b88378 | rmar3a | 27 March 2020, 19:13:38 UTC | Added database logging to RuntimeLogger. - Runtime information related to contingency table generation is now written to a "CallLogs" table within the "_BN" database. - Added methods to RuntimeLogger for writing logging information to the "CallLogs" table. - Created a new script, logging.sql, that creates the "CallLogs" table, which is used to help log the runtimes for each method call that is related to contingency table generation. - Renamed the method BuildCT_Pvars() to buildPVarsCounts() to match the naming format of the method buildRChainCounts(). - Updated the dbdump.sh script so that it ignores the CallLogs table when generating the mysql-extraction.txt file since the runtime values are expected to differ slightly between FactorBase runs. | 27 March 2020, 19:13:38 UTC |
| 9929c4f | oschulte | 26 March 2020, 22:26:14 UTC | Merge pull request #200 from sfu-cl-lab/single-connection Use a single Connection object in CountsManager.java. | 26 March 2020, 22:26:14 UTC |
| 8d9beaa | rmar3a | 26 March 2020, 18:57:26 UTC | Use a single Connection object in CountsManager.java. - Removed the separate Connection objects for the "_BN" and "_CT" databases and replaced them with a single Connection object. - The setCatalog() method is now used instead to change between the two databases as needed. - The buildRChainCounts() method now takes in the name of the database to create the counts tables in since the target could be the "_CT" or the "_global_counts" database. Downstream code was updated appropriately. - Removed an unused Statement object. Note: This change will hopefully help prevent the following issue from happening: Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. | 26 March 2020, 20:27:26 UTC |
| fac04a4 | oschulte | 25 March 2020, 17:08:43 UTC | Merge pull request #197 from sfu-cl-lab/hybrid Hybrid method for generating the contingency tables. | 25 March 2020, 17:08:43 UTC |
| 1194363 | oschulte | 25 March 2020, 17:08:00 UTC | Merge pull request #198 from sfu-cl-lab/minor-changes Minor changes | 25 March 2020, 17:08:00 UTC |
| 94f312d | rmar3a | 25 March 2020, 01:47:22 UTC | The server time zone is specified to PST for database connection strings. - When the server time zone is not specified, the value of PDT is used sometimes and isn't valid and causes FactorBase to crash. - The server time zone is now specified to PST. | 25 March 2020, 01:47:22 UTC |
| 1b10596 | rmar3a | 25 March 2020, 01:43:41 UTC | Updated RuntimeLogger to conform to other Utility classes. - Made RuntimeLogger a final class and added a private constructor so that it is like the other Utility classes in the project and follows better practices. | 25 March 2020, 01:43:41 UTC |
| c888794 | rmar3a | 25 March 2020, 00:58:20 UTC | Hybrid method for generating the contingency tables. - When hybrid counting is used, the local "_counts" tables for RChains are created by projecting the results from the global "_counts" tables in the "_global_counts" database. - Adjusted logic in the generateCountsTable() method so that they are projected from the "_global_counts" database when specified to. - Created a method to remove aliasing from SELECT statements when projecting from columns. - Created a new enum to help guide the execution of FactorBase depending on what counting strategy is specified. - The counting strategy can be specified in the configuration file using the "CountingStrategy" settings. The current valid values for this setting are: 0 - Precount 1 - Ondemand 2 - Hybrid | 25 March 2020, 01:42:33 UTC |
| 1f3e5f4 | oschulte | 20 March 2020, 22:58:28 UTC | Merge pull request #192 from sfu-cl-lab/improved-query Added an extra condition when creating the EntityTables. | 20 March 2020, 22:58:28 UTC |
| a1baf3f | oschulte | 20 March 2020, 22:57:45 UTC | Merge pull request #196 from sfu-cl-lab/update-connectorJ Update MySQL Connector/J to the Latest Version | 20 March 2020, 22:57:45 UTC |
| 1441b74 | rmar3a | 31 January 2020, 00:24:02 UTC | Removed the white-space for RNode IDs generated in certain tables. - Removed the white-space for the RNode IDs in the RNodes_MO_NotSelf and RNodes_MO_Self tables. | 20 March 2020, 22:28:28 UTC |
| 2be224b | rmar3a | 30 January 2020, 23:28:10 UTC | Added an extra condition when creating the RNodes_MO_NotSelf. - Duplicate table names could appear in the RNodes_MO_NotSelf table due to things like secondary keys, so a restriction is placed to ensure that each RNode ID is listed at most once. | 20 March 2020, 22:28:28 UTC |
