<> One , Tool introduction

<>Kettle brief introduction

be based on JAVA Of ETL tool , Support graphical GUI Design interface , Then it can flow in the form of workflow , Doing some simple or complex data extraction , Quality inspection , Data cleaning , data conversion , Data filtering and other aspects have a relatively stable performance

<>Sqoop brief introduction

Apache Open source software , Mainly used in HADOOP(Hive) And traditional database (mysql,postgresql…) Data transfer between .
It is suitable for mass data transmission between relational databases which can communicate directly with big data cluster .

<> Two , contrast

function KettleSqoop
Domain data extraction , transformation , Loading relational and non relational database data migration
Input relational database ,HDFS,Hbase,Excel,HL7,JSON,RSS, text file , And so on , Non relational database
Output relational database ,Hbase,HDFS,Excel,CSV, And so on , Non relational database
Hadoop Integration external tools , You need to install the corresponding version of the plug-in , Only popular Hadoop The distribution belongs to Hadoop Ecosphere , Ready to use
The applicable data volume is 100000 , million , Tens of millions
support system Linux,Windows,UnixLinux
Interaction has a graphical interface, no graphical interface
Low level multithreading improves efficiency MapReduce
<> Three , summary

If the requirement is just to put the relational database data (Oracle,MySQL) Migrating to a non relational database (HDFS,Hbase,Hive), Recommended Sqoop tool , Enough to meet demand

If it is a relational database of different kinds (Oracle,MySQL,SQL
server) Integration into the same relational database , as MySQL. Recommended Kettle, Yes GUI The interface is easy to operate .