Difference between revisions of "Howto/MrPlowDataMigration"
(→Run Data Migration) |
|||
| (2 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| + | |||
| + | ==Mr Plow Data Migration== | ||
| + | |||
| + | From RocksDB to Atomic File DB. | ||
| + | |||
| + | == Overview == | ||
Before 2021, [[MrPlow]] was suggested to run with [[ConfigOption/db_type]] of rocksdb. RocksDB has great performance and is generally fairly reliable, but it does occasionally break in various ways. For a node, that is not a big problem. You can always delete a node DB and then resync. | Before 2021, [[MrPlow]] was suggested to run with [[ConfigOption/db_type]] of rocksdb. RocksDB has great performance and is generally fairly reliable, but it does occasionally break in various ways. For a node, that is not a big problem. You can always delete a node DB and then resync. | ||
| Line 7: | Line 13: | ||
This guide is for how to switch a mining pool from RocksDB to Atomic File while maintaining the share state. | This guide is for how to switch a mining pool from RocksDB to Atomic File while maintaining the share state. | ||
| + | ==Steps== | ||
| + | |||
| + | ===Stop Pool=== | ||
| + | Stop Pool. The RocksDB database must be closed. | ||
| + | |||
| + | ===Run Data Migration=== | ||
| + | |||
| + | The data migration tool is in the source. A <code>bazel build :all</code> will create it. | ||
| + | |||
| + | The command is MrPlowDataMigrate and takes two parameters, the path to the existing RocksDB data and the path to the new Atomic File DB. | ||
| + | The Atomic File DB path doesn't need to exist, it will be created. | ||
| + | |||
| + | <pre> | ||
| + | $ bazel-bin/MrPlowDataMigrate /home/nerd/plow.db /home/nerd/plow.file.db | ||
| + | Migrating data from rocksdb: /home/nerd/plow.db | ||
| + | Migrating data to atomic file db: /home/nerd/plow.file.db | ||
| + | Dec 31, 2020 10:10:19 PM snowblossom.lib.db.rocksdb.JRocksDB <init> | ||
| + | INFO: Loading RocksDB with path /home/nerd/plow.db | ||
| + | Dec 31, 2020 10:10:19 PM snowblossom.lib.db.atomicfile.AtomicFileDB <init> | ||
| + | INFO: Loading AtomicFileDB with path /home/nerd/plow.file.db | ||
| + | Dec 31, 2020 10:10:19 PM snowblossom.miner.plow.DataMigrate main | ||
| + | INFO: Have source PPLNS state with 270 entries | ||
| + | Dec 31, 2020 10:10:30 PM snowblossom.miner.plow.DataMigrate main | ||
| + | INFO: Saved found block list: 68049 | ||
| + | Dec 31, 2020 10:10:30 PM snowblossom.miner.plow.DataMigrate main | ||
| + | INFO: PPLNS state saved | ||
| + | Dec 31, 2020 10:10:30 PM snowblossom.miner.plow.DataMigrate main | ||
| + | INFO: Migration complete | ||
| + | Dec 31, 2020 10:10:30 PM snowblossom.lib.db.rocksdb.JRocksDB close | ||
| + | INFO: RocksDB flush started | ||
| + | Dec 31, 2020 10:10:30 PM snowblossom.lib.db.rocksdb.JRocksDB close | ||
| + | INFO: RocksDB flush completed | ||
| + | </pre> | ||
| + | |||
| + | The migration does sanity checks. It will only proceed if the source does have a PPLNS state and the destination does not. | ||
| + | |||
| + | ===Update Config=== | ||
| + | |||
| + | Update your plow config. Set: | ||
| + | |||
| + | <code> | ||
| + | db_type=atomic_file | ||
| + | </code> | ||
| + | |||
| + | and | ||
| + | <code>db_path=<new path></code> | ||
| + | to be the new location you created in the migration step. | ||
| + | ===Restart Pool=== | ||
| + | You should be good to go. | ||
[[Category:HowTo]] | [[Category:HowTo]] | ||
Latest revision as of 06:26, 1 January 2021
Contents
Mr Plow Data Migration
From RocksDB to Atomic File DB.
Overview
Before 2021, MrPlow was suggested to run with ConfigOption/db_type of rocksdb. RocksDB has great performance and is generally fairly reliable, but it does occasionally break in various ways. For a node, that is not a big problem. You can always delete a node DB and then resync.
However, for things that need to be persisted, it isn't good enough. So there is a new DB type of 'atomic_file' which uses one file per record. For things like MrPlow that has a small number of records but the records are very important, that is a better fit.
This guide is for how to switch a mining pool from RocksDB to Atomic File while maintaining the share state.
Steps
Stop Pool
Stop Pool. The RocksDB database must be closed.
Run Data Migration
The data migration tool is in the source. A bazel build :all will create it.
The command is MrPlowDataMigrate and takes two parameters, the path to the existing RocksDB data and the path to the new Atomic File DB. The Atomic File DB path doesn't need to exist, it will be created.
$ bazel-bin/MrPlowDataMigrate /home/nerd/plow.db /home/nerd/plow.file.db Migrating data from rocksdb: /home/nerd/plow.db Migrating data to atomic file db: /home/nerd/plow.file.db Dec 31, 2020 10:10:19 PM snowblossom.lib.db.rocksdb.JRocksDB <init> INFO: Loading RocksDB with path /home/nerd/plow.db Dec 31, 2020 10:10:19 PM snowblossom.lib.db.atomicfile.AtomicFileDB <init> INFO: Loading AtomicFileDB with path /home/nerd/plow.file.db Dec 31, 2020 10:10:19 PM snowblossom.miner.plow.DataMigrate main INFO: Have source PPLNS state with 270 entries Dec 31, 2020 10:10:30 PM snowblossom.miner.plow.DataMigrate main INFO: Saved found block list: 68049 Dec 31, 2020 10:10:30 PM snowblossom.miner.plow.DataMigrate main INFO: PPLNS state saved Dec 31, 2020 10:10:30 PM snowblossom.miner.plow.DataMigrate main INFO: Migration complete Dec 31, 2020 10:10:30 PM snowblossom.lib.db.rocksdb.JRocksDB close INFO: RocksDB flush started Dec 31, 2020 10:10:30 PM snowblossom.lib.db.rocksdb.JRocksDB close INFO: RocksDB flush completed
The migration does sanity checks. It will only proceed if the source does have a PPLNS state and the destination does not.
Update Config
Update your plow config. Set:
db_type=atomic_file
and
db_path=<new path>
to be the new location you created in the migration step.
Restart Pool
You should be good to go.
