Test server

Hardware information :
[root@catdb ~]# cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l 2
[root@catdb ~]# cat /proc/cpuinfo| grep "cpu cores"| uniq cpu cores : 6
[root@catdb ~]# cat /proc/cpuinfo| grep "processor"| wc -l 12 [root@catdb ~]#
cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c 12 Intel(R) Xeon(R) CPU
E5-2603 v3 @ 1.60GHz [root@catdb ~]# free -m total used free shared buff/cache
available Mem: 15757 5056 938 1206 9762 9047 Swap: 32767 6765 26002
io performance :
[root@catdb bin]# time dd if=/root/ubuntu.tar of=test.file 2553093+0 records
in 2553093+0 records out 1307183616 bytes (1.3 GB) copied, 7.89118 s, 166 MB/s
real 0m7.906s user 0m0.454s sys 0m5.539s
Table information :       
size:    1.12G  num_rows:6,015,341 columns: 21
Test scenario : Full extraction and loading to the library . meanwhile kettle Follow datax All are executed on the server .

Testing process :
kettle Of log: 2019/12/20 11:02:44 - Spoon - Transformation opened. 2019/12/20
11:02:44 - Spoon - Launching transformation [rcattest01]... 2019/12/20 11:02:44
- Spoon - Started the transformation execution.   ......... 2019/12/20 11:05:33
- Table input .0 - Finished reading query, closing connection. 2019/12/20 11:05:33 -
Table input .0 - Finished processing (I=6015341, O=0, R=0, W=6015341, U=0, E=0)
2019/12/20 11:05:33 - Table output .0 - Finished processing (I=0, O=6015341, R=6015341,
W=6015341, U=0, E=0) 2019/12/20 11:05:33 - Spoon - The transformation has
finished!! dataxde Of log:    ......... 2019-12-20 11:19:24.851 [job-0] INFO
 JobContainer -  Mission start time                    : 2019-12-20 11:17:13 End of mission        
           : 2019-12-20 11:19:24 Total task time                    :              
 131s Average task traffic                    :            8.90MB/s Record write speed                  
 :          46271rec/s Total number of read records                    :             6015341 Total number of read / write failures  
                 :                   0
test result :

        kettle time consuming 167 second    VS  datax time consuming 131 second .

        600 Ten thousand pieces in total 1.12G Size table ,datax than kettle fast 36 second .

Technology
©2020 ioDraw All rights reserved
LeetCode quick get start ① —— Array series ( Often asked in an interview , Suggested collection ) Apple iPhone 12 Price leakage : The official highest price is close to 1 Ten thousand yuan ( Essence )2020 year 6 month 26 day C# Class library Ip Address help class spark.sql.shuffle.partitions and spark.default.parallelism The difference between flutter study -- Search box NVIDIA official apology : Robots are sold out RTX 3080django Do not close CSRF middleware , Custom through CSRF Tested post request 【 Deep learning 】 Target detection in progress IOU The concept and calculation of ( Essence )2020 year 6 month 26 day C# Class library DataTable( Extension method ) Can Huawei Hongmeng system and Android , Apples compete against each other ?