With the size of the Ethereum block chain always increasing, it is important to occasionally back up your copy of the ledger. In case your PC crashes, you will not have to start downloading the block chain from block zero.
Shut down any running geth processes
Make a backup of your keystore:
C:\Desktop>copy %APPDATA%\Ethereum\keystore\* .
C:\AppData\Roaming\Ethereum\keystore\UTC--2017-blah-blah-9
1 file(s) copied.
Mentally make a backup of the password for your keystore.
Back up the downloaded Ethereum blockchain:
C:\Users\Desktop>geth export ethereum_blockchain_20170830
INFO
[08-30|12:38:28] Allocated cache and file handles
database=C:\Users\AppData\Roaming\Ethereum\geth\chaindata
cache=128 handles=1024
INFO [08-30|12:38:30] Loaded most recent local header number=4220763 hash=39eda3…fd5e71 td=810099527882589796805
INFO [08-30|12:38:30] Loaded most recent local full block number=4220763 hash=39eda3…fd5e71 td=810099527882589796805
INFO [08-30|12:38:30] Loaded most recent local fast block number=4220763 hash=39eda3…fd5e71 td=810099527882589796805
INFO [08-30|12:38:30] Exporting blockchain file=ethereum_blockchain_20170830
INFO [08-30|12:38:30] Exporting batch of blocks count=4220764
INFO [08-30|12:49:53] Exported blockchain file=ethereum_blockchain_20170830
Export done in 11m22.8688061s
Showing posts with label Ethereum. Show all posts
Showing posts with label Ethereum. Show all posts
Saturday, September 02, 2017
Tuesday, June 07, 2016
Tuning I/O for Ethereum
At a high level, the I/O request goes from the running program to the operating system. Then from the operating system to the appropriate file system driver, then to the disk controller which writes to the disk.
It is usually important to match the I/O length (the amount of data being written) in each step of the process. Disks and the disk controllers typically write large stripes of data, such as 512KB. The file system may be set up with RAID, which can have stripes of 512KB or 1 MB. The operating system may default to a size of 16 KB (NTFS default has been 4 KB). To complicate this further, the operating system may implement file compression, which can pack more data into each I/O. And remember the filesystem may have logging, and may also frequently update inode access times which can cause contention.
It is typically difficult to change RAID settings after it has been implemented. RAID 5 will often be slow. It is typically impossible to change disk controller settings.
Therefore, let's concentrate on tuning a high I/O workload with commonly available settings in the filesystem and the program.
Increase the filesystem block size to a reasonably large value such as 64 KB. This is set when formatting a partition. While larger block sizes can lead to some wasted disk space, this is not typically a problem with modern large disks.
For tuning the program, if possible increase the program's (database or geth) cache size and match the logical I/O to the filesystem I/O size. For geth, it is possible to set the cache size with the --cache flag:
.\geth.exe --cache 512
The default geth cache is 16 MB, so 512 MB is a large increase. Set this lower than the amount of real RAM in the system, to avoid creating a swapping situation. Monitor this with Task Manager, using the following examples. Note 1.6 GB of real RAM is available to avoid swapping, which is a safe margin of error in case the program data increases.
Using the geth default of 16 MB, the maximum observed I/O was about 3 MB per second. Using geth's cache of 512 MB increased the maximum I/O to about 12 MB per second, a significant increase.
It is usually important to match the I/O length (the amount of data being written) in each step of the process. Disks and the disk controllers typically write large stripes of data, such as 512KB. The file system may be set up with RAID, which can have stripes of 512KB or 1 MB. The operating system may default to a size of 16 KB (NTFS default has been 4 KB). To complicate this further, the operating system may implement file compression, which can pack more data into each I/O. And remember the filesystem may have logging, and may also frequently update inode access times which can cause contention.
It is typically difficult to change RAID settings after it has been implemented. RAID 5 will often be slow. It is typically impossible to change disk controller settings.
Therefore, let's concentrate on tuning a high I/O workload with commonly available settings in the filesystem and the program.
Increase the filesystem block size to a reasonably large value such as 64 KB. This is set when formatting a partition. While larger block sizes can lead to some wasted disk space, this is not typically a problem with modern large disks.
For tuning the program, if possible increase the program's (database or geth) cache size and match the logical I/O to the filesystem I/O size. For geth, it is possible to set the cache size with the --cache flag:
.\geth.exe --cache 512
The default geth cache is 16 MB, so 512 MB is a large increase. Set this lower than the amount of real RAM in the system, to avoid creating a swapping situation. Monitor this with Task Manager, using the following examples. Note 1.6 GB of real RAM is available to avoid swapping, which is a safe margin of error in case the program data increases.
Using the geth default of 16 MB, the maximum observed I/O was about 3 MB per second. Using geth's cache of 512 MB increased the maximum I/O to about 12 MB per second, a significant increase.
Wednesday, April 27, 2016
How to mine Ethereum on Microsoft Windows 64 bit - benchmark the miner and mine
These instructions have been tested on Microsoft Windows 10 64 bit, 4GB RAM, with an ATI video card.
In a prior post, the geth software which communicates with the Ethereum network was downloaded and installed. An account address was created and the current block chain was downloaded. The mining software was downloaded and installed.
In this post, we will test the mining software by benchmarking both CPU mining and GPU mining.
Two command prompt windows will be opened, one for geth and one for ethminer.
Open a cmd prompt.
Go to the geth software directory.
Start geth by typing: .\geth --rpc
The block chain will be synchronized, which should take a few minutes to download the latest blocks.
Open another cmd prompt.
Go to the ethminer directory: cd %PROGRAMFILES%\Ethereum <release number>\Release
Ethminer communicates with geth, so ensure it is still running in the other cmd window. The first time ethminer is run, it will create a "DAG" file. The DAG creation process conveniently shows percent complete.
Start ethminer to create the DAG file.
C:\Program Files\Ethereum 0.9.41\Release>.\ethminer
miner 13:13:26|main Getting work package...
JSON-RPC problem. Probably couldn't connect. Retrying in 1...
miner 13:13:28|main Getting work package...
miner 13:13:28|main Grabbing DAG for #0016bedd…
DAG 13:13:55|main Generating DAG file. Progress: 0 %
Creating DAG. 0% done...DAG 13:14:08|main Generating DAG file. Progress: 1 %
Creating DAG. 1% done...
Start ethminer in benchmark mode (-M) and benchmark CPU mining.
Benchmarking on platform: CPU
Preparing DAG...
Warming up...
Trial 1... 377068
Trial 2... 381018
Trial 3... 385818
Trial 4... 386728
Trial 5... 388705
min/mean/max: 377068/383867/388705 H/s
inner mean: 384521 H/s
Phoning home to find world ranking...
Start ethminer in benchmark mode (-M) and benchmark GPU (-G) mining.
.\ethminer -G -M
If the miner does not start, there are some troubleshooting steps. A common failure message is "No GPU device with sufficient memory was found. Can't GPU mine. Remove the -G argument"
List the GPU devices: .\ethminer --list-devices
Listing OpenCL devices.
FORMAT: [deviceID] deviceName
[0] Bonaire
CL_DEVICE_TYPE: GPU
CL_DEVICE_GLOBAL_MEM_SIZE: 2147483648
CL_DEVICE_MAX_MEM_ALLOC_SIZE: 1409286144
CL_DEVICE_MAX_WORK_GROUP_SIZE: 256
Try to specify the GPU: .\ethminer -G -M --opencl-device 0
Try to specify a different OpenCL platform implementation: .\ethminer -G --opencl-platform 1 -M
Found suitable OpenCL device [Bonaire] with 2147483648 bytes of GPU memory
Benchmarking on platform: GPU
Preparing DAG...
i 22:35:42|gpumineWra0r m iwnogr kuLpo.o.p.
0 #00000000… #00000000…
i 22:35:42|gpuminer0 Initialising miner...
Using platform: AMD Accelerated Parallel Processing
Using device: Bonaire(OpenCL 2.0 AMD-APP (2004.6))
Printing program log
Creating one big buffer for the DAG
Loading single big chunk kernels
Mapping one big chunk.
Creating buffer for header.
Creating mining buffer 0
Creating mining buffer 1
Trial 1... 4801172
Trial 2... 4886838
Trial 3... 4972448
Trial 4... 4886838
Trial 5... 4974103
min/mean/max: 4801172/4904279/4974103 H/s
inner mean: 4944463 H/s
Phoning home to find world ranking...
Start mining with whichever flags work on your platform:
.\ethminer -G
.\ethminer -G --opencl-platform 1
In a prior post, the geth software which communicates with the Ethereum network was downloaded and installed. An account address was created and the current block chain was downloaded. The mining software was downloaded and installed.
In this post, we will test the mining software by benchmarking both CPU mining and GPU mining.
Two command prompt windows will be opened, one for geth and one for ethminer.
Open a cmd prompt.
Go to the geth software directory.
Start geth by typing: .\geth --rpc
The block chain will be synchronized, which should take a few minutes to download the latest blocks.
Open another cmd prompt.
Go to the ethminer directory: cd %PROGRAMFILES%\Ethereum <release number>\Release
Ethminer communicates with geth, so ensure it is still running in the other cmd window. The first time ethminer is run, it will create a "DAG" file. The DAG creation process conveniently shows percent complete.
Start ethminer to create the DAG file.
C:\Program Files\Ethereum 0.9.41\Release>.\ethminer
miner 13:13:26|main Getting work package...
JSON-RPC problem. Probably couldn't connect. Retrying in 1...
miner 13:13:28|main Getting work package...
miner 13:13:28|main Grabbing DAG for #0016bedd…
DAG 13:13:55|main Generating DAG file. Progress: 0 %
Creating DAG. 0% done...DAG 13:14:08|main Generating DAG file. Progress: 1 %
Creating DAG. 1% done...
Start ethminer in benchmark mode (-M) and benchmark CPU mining.
Benchmarking on platform: CPU
Preparing DAG...
Warming up...
Trial 1... 377068
Trial 2... 381018
Trial 3... 385818
Trial 4... 386728
Trial 5... 388705
min/mean/max: 377068/383867/388705 H/s
inner mean: 384521 H/s
Phoning home to find world ranking...
Start ethminer in benchmark mode (-M) and benchmark GPU (-G) mining.
.\ethminer -G -M
If the miner does not start, there are some troubleshooting steps. A common failure message is "No GPU device with sufficient memory was found. Can't GPU mine. Remove the -G argument"
List the GPU devices: .\ethminer --list-devices
Listing OpenCL devices.
FORMAT: [deviceID] deviceName
[0] Bonaire
CL_DEVICE_TYPE: GPU
CL_DEVICE_GLOBAL_MEM_SIZE: 2147483648
CL_DEVICE_MAX_MEM_ALLOC_SIZE: 1409286144
CL_DEVICE_MAX_WORK_GROUP_SIZE: 256
Try to specify the GPU: .\ethminer -G -M --opencl-device 0
Try to specify a different OpenCL platform implementation: .\ethminer -G --opencl-platform 1 -M
Found suitable OpenCL device [Bonaire] with 2147483648 bytes of GPU memory
Benchmarking on platform: GPU
Preparing DAG...
i 22:35:42|gpumineWra0r m iwnogr kuLpo.o.p.
0 #00000000… #00000000…
i 22:35:42|gpuminer0 Initialising miner...
Using platform: AMD Accelerated Parallel Processing
Using device: Bonaire(OpenCL 2.0 AMD-APP (2004.6))
Printing program log
Creating one big buffer for the DAG
Loading single big chunk kernels
Mapping one big chunk.
Creating buffer for header.
Creating mining buffer 0
Creating mining buffer 1
Trial 1... 4801172
Trial 2... 4886838
Trial 3... 4972448
Trial 4... 4886838
Trial 5... 4974103
min/mean/max: 4801172/4904279/4974103 H/s
inner mean: 4944463 H/s
Phoning home to find world ranking...
Start mining with whichever flags work on your platform:
.\ethminer -G
.\ethminer -G --opencl-platform 1
Monday, April 25, 2016
How to mine Ethereum on Microsoft Windows 64 bit - ethminer
These instructions have been tested on Microsoft Windows 10 64 bit, 4GB RAM, with an ATI video card.
In a prior post, the geth software which communicates with the Ethereum network, was downloaded and installed. An account address was created and the current block chain was downloaded.
In this post, the block miner ethminer will be downloaded and installed.
Download the latest Ethereum miner at https://build.ethdev.com/builds/Windows%20C%2B%2B%20develop%20branch/Ethereum-win64-latest.exe
Unzip the files. By default, the files will go to the %PROGRAMFILES%\Ethereum <release number>
The Ethereum miner has been downloaded and installed.
The next step is to test the miner.
In a prior post, the geth software which communicates with the Ethereum network, was downloaded and installed. An account address was created and the current block chain was downloaded.
In this post, the block miner ethminer will be downloaded and installed.
Download the latest Ethereum miner at https://build.ethdev.com/builds/Windows%20C%2B%2B%20develop%20branch/Ethereum-win64-latest.exe
Unzip the files. By default, the files will go to the %PROGRAMFILES%\Ethereum <release number>
The Ethereum miner has been downloaded and installed.
The next step is to test the miner.
Friday, April 22, 2016
How to mine Ethereum on Microsoft Windows 64 bit - geth
These instructions have been tested on Microsoft Windows 10 64 bit, 4GB RAM, with an ATI video card.
Geth is software which communicates with the Ethereum network, a block chain crypto-currency and global programming network.
Download the latest version of geth at https://github.com/ethereum/go-ethereum/releases
Put geth in a convenient directory, like c:
Open a command prompt by pressing Windows button and typing: cmd
Navigate to the geth directory and unzip the files.
Using a strong password which you will remember, create an account address.
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>.\geth account new
Your new account is locked with a password. Please give a password. Do not forget this password.
Passphrase:
Repeat Passphrase:
Address: {long_address_string_____________________}
The account keys are stored in <DATADIR>/keystore.
View the new account key in directory: cd %APPDATA%\Ethereum
Display the geth version.
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>.\geth version
Geth
Version: 1.3.6
Protocol Versions: [63 62 61]
Network Id: 1
Go Version: go1.5.1
OS: windows
GOPATH=
GOROOT=c:\go
Start geth to download and synchronize the block chain.
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>.\geth --rpc
I0411 16:54:24.451793 4296 database.go:71] Alloted 16MB cache to C:\Users\mswindows_username\AppData\Roaming\Ethereum\chaindata
I0411 16:54:25.233804 4296 database.go:71] Alloted 16MB cache to C:\Users\mswindows_username\AppData\Roaming\Ethereum\dapp
I0411 16:54:25.270260 4296 backend.go:314] Protocol Versions: [63 62 61], Network Id: 1
I0411 16:54:25.351417 4296 backend.go:362] Blockchain DB Version: 3
I0411 16:54:25.582006 4296 blockchain.go:214] Last header: #99215 [babce72f…] TD=166437631468780593
I0411 16:54:25.585960 4296 blockchain.go:215] Last block: #99215 [babce72f…] TD=166437631468780593
I0411 16:54:25.592422 4296 blockchain.go:216] Fast block: #99215 [babce72f…] TD=166437631468780593
I0411 16:54:25.787411 4296 cmd.go:115] Starting Geth/v1.3.6/windows/go1.5.1
I0411 16:54:25.800840 4296 server.go:311] Starting Server
I0411 16:54:27.043156 4296 udp.go:212] Listening, enode://e4f5fd7e13e73e227bbe2f5ca4001f4872ef24379199fe290b99df65bad9dc6f98248dfd0554bffc318973dc7eae21a17a934199e869133201af7e8331418b51@[::]:30303
I0411 16:54:27.052664 4296 backend.go:526] Server started
I0411 16:54:27.052664 4296 server.go:552] Listening on [::]:30303
I0411 16:54:27.073593 4296 ipc.go:112] IPC service started (\\.\pipe\geth.ipc)
I0411 16:54:47.054104 4296 downloader.go:288] Block synchronisation started
I0411 16:56:24.358034 4296 blockchain.go:1251] imported 256 block(s) (0 queued 0 ignored) including 58 txs in 7.7356549s. #99471 [7378a4ef / 0ed10e27]
I0411 16:56:28.799755 4296 blockchain.go:1251] imported 256 block(s) (0 queued 0 ignored) including 51 txs in 4.4371818s. #99727 [5b7e1e92 / 70ef8fdb]
Geth will acquire blocks in chunks of 256 (the first chunk can take minutes to appear). In the geth output, you will typically see low "txs" numbers and importing one block at a time when the block chain is synchronized.
With a web broswer, go to https://ethstats.net/ and look at "BEST BLOCK". Geth will be done downloading the block chain when your machine shows the "best block" number.
After the block chain is synchronized, end the geth process by pressing: ctrl-c
I0415 12:45:31.430682 4296 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 0 txs in 8.981ms. #1342669 [ebc34c57 / ebc34c57]
I0415 12:45:35.800612 4296 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 6 txs in 42.0009ms. #1342670 [9e7e42f5 / 9e7e42f5]
I0415 12:46:00.582403 4296 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 5 txs in 29.5749ms. #1342671 [0f1555a6 / 0f1555a6]
I0415 12:46:17.510895 4296 cmd.go:124] Got interrupt, shutting down...
I0415 12:46:17.617873 4296 blockchain.go:690] Chain manager stopped
I0415 12:46:17.621902 4296 handler.go:192] Stopping ethereum protocol handler...
I0415 12:46:17.624903 4296 handler.go:202] Ethereum protocol handler stopped
I0415 12:46:17.628907 4296 tx_pool.go:156] Transaction pool stopped
I0415 12:46:17.650008 4296 backend.go:630] Automatic pregeneration of ethash DAG OFF (ethash dir: C:\Users\mswindows_username\AppData\Ethash)
I0415 12:46:17.719083 4296 database.go:158] closed db: C:\Users\mswindows_username\AppData\Roaming\Ethereum\chaindata
I0415 12:46:17.728089 4296 database.go:158] closed db: C:\Users\mswindows_username\AppData\Roaming\Ethereum\dapp
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>
List the new account.
.\geth account list
Using geth console, verify there are zero rewards on the new account address.
.\geth console
web3.fromWei(eth.getBalance(eth.coinbase), "ether")
exit
Geth has now been downloaded, an account has been created, and the existing block chain has been downloaded.
The next step is to download the Ethereum miner.
Geth is software which communicates with the Ethereum network, a block chain crypto-currency and global programming network.
Download the latest version of geth at https://github.com/ethereum/go-ethereum/releases
Put geth in a convenient directory, like c:
Open a command prompt by pressing Windows button and typing: cmd
Navigate to the geth directory and unzip the files.
Using a strong password which you will remember, create an account address.
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>.\geth account new
Your new account is locked with a password. Please give a password. Do not forget this password.
Passphrase:
Repeat Passphrase:
Address: {long_address_string_____________________}
The account keys are stored in <DATADIR>/keystore.
View the new account key in directory: cd %APPDATA%\Ethereum
Display the geth version.
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>.\geth version
Geth
Version: 1.3.6
Protocol Versions: [63 62 61]
Network Id: 1
Go Version: go1.5.1
OS: windows
GOPATH=
GOROOT=c:\go
Start geth to download and synchronize the block chain.
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>.\geth --rpc
I0411 16:54:24.451793 4296 database.go:71] Alloted 16MB cache to C:\Users\mswindows_username\AppData\Roaming\Ethereum\chaindata
I0411 16:54:25.233804 4296 database.go:71] Alloted 16MB cache to C:\Users\mswindows_username\AppData\Roaming\Ethereum\dapp
I0411 16:54:25.270260 4296 backend.go:314] Protocol Versions: [63 62 61], Network Id: 1
I0411 16:54:25.351417 4296 backend.go:362] Blockchain DB Version: 3
I0411 16:54:25.582006 4296 blockchain.go:214] Last header: #99215 [babce72f…] TD=166437631468780593
I0411 16:54:25.585960 4296 blockchain.go:215] Last block: #99215 [babce72f…] TD=166437631468780593
I0411 16:54:25.592422 4296 blockchain.go:216] Fast block: #99215 [babce72f…] TD=166437631468780593
I0411 16:54:25.787411 4296 cmd.go:115] Starting Geth/v1.3.6/windows/go1.5.1
I0411 16:54:25.800840 4296 server.go:311] Starting Server
I0411 16:54:27.043156 4296 udp.go:212] Listening, enode://e4f5fd7e13e73e227bbe2f5ca4001f4872ef24379199fe290b99df65bad9dc6f98248dfd0554bffc318973dc7eae21a17a934199e869133201af7e8331418b51@[::]:30303
I0411 16:54:27.052664 4296 backend.go:526] Server started
I0411 16:54:27.052664 4296 server.go:552] Listening on [::]:30303
I0411 16:54:27.073593 4296 ipc.go:112] IPC service started (\\.\pipe\geth.ipc)
I0411 16:54:47.054104 4296 downloader.go:288] Block synchronisation started
I0411 16:56:24.358034 4296 blockchain.go:1251] imported 256 block(s) (0 queued 0 ignored) including 58 txs in 7.7356549s. #99471 [7378a4ef / 0ed10e27]
I0411 16:56:28.799755 4296 blockchain.go:1251] imported 256 block(s) (0 queued 0 ignored) including 51 txs in 4.4371818s. #99727 [5b7e1e92 / 70ef8fdb]
Geth will acquire blocks in chunks of 256 (the first chunk can take minutes to appear). In the geth output, you will typically see low "txs" numbers and importing one block at a time when the block chain is synchronized.
With a web broswer, go to https://ethstats.net/ and look at "BEST BLOCK". Geth will be done downloading the block chain when your machine shows the "best block" number.
After the block chain is synchronized, end the geth process by pressing: ctrl-c
I0415 12:45:31.430682 4296 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 0 txs in 8.981ms. #1342669 [ebc34c57 / ebc34c57]
I0415 12:45:35.800612 4296 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 6 txs in 42.0009ms. #1342670 [9e7e42f5 / 9e7e42f5]
I0415 12:46:00.582403 4296 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 5 txs in 29.5749ms. #1342671 [0f1555a6 / 0f1555a6]
I0415 12:46:17.510895 4296 cmd.go:124] Got interrupt, shutting down...
I0415 12:46:17.617873 4296 blockchain.go:690] Chain manager stopped
I0415 12:46:17.621902 4296 handler.go:192] Stopping ethereum protocol handler...
I0415 12:46:17.624903 4296 handler.go:202] Ethereum protocol handler stopped
I0415 12:46:17.628907 4296 tx_pool.go:156] Transaction pool stopped
I0415 12:46:17.650008 4296 backend.go:630] Automatic pregeneration of ethash DAG OFF (ethash dir: C:\Users\mswindows_username\AppData\Ethash)
I0415 12:46:17.719083 4296 database.go:158] closed db: C:\Users\mswindows_username\AppData\Roaming\Ethereum\chaindata
I0415 12:46:17.728089 4296 database.go:158] closed db: C:\Users\mswindows_username\AppData\Roaming\Ethereum\dapp
C:\Geth-Win64-20160401105807-1.3.6-9e323d6>
List the new account.
.\geth account list
Using geth console, verify there are zero rewards on the new account address.
.\geth console
web3.fromWei(eth.getBalance(eth.coinbase), "ether")
exit
Geth has now been downloaded, an account has been created, and the existing block chain has been downloaded.
The next step is to download the Ethereum miner.
Subscribe to:
Posts (Atom)