Dixer¶
What is Dixer?¶
Dixer is a tool to that can be used to execute a wide range of ETL, data transformation and data migration task. Also includes jobs for performing workflow functions like sending email messages, file and FTP operations, data sources and destinations, delay, make a HTTP or SOAP request and more.
Why Dixer?¶
- Automates Administrative Functions and Data Loading.
- It can load millions of rows from one data source to another in very few minutes.
- Dixer eliminates the need of hardcore programmers.
- Dixer offers simple error and event handling.
- Easy workflow with Jobs.
- Multiplatform.
- No need installation and dependences.
- Ease and speed of implementation.
- Standardized data transformation.
- Integrates with almost all bigs relational databases (ORACLE, SAP HANA DB, IBM DB2, MySQL, MSSQL, PostgreSQL and more).
Motivation¶
I work with SSIS and Talend, these are great tools to automate data loading and to do data transformation, but there are to big... TOO BIG!
Some requeriments doesn't require all magical features of these tools... a simple loading, or send a email, etc... is fine but you need the complete installation of Talend Studio or SSDT to create the package, this is why I created Dixer, a simple binary to cover a lot of possibilities!
Also is more simple, Do you want pass a lot of rows from Oracle to MSSQL? In SSIS and Talend you need to do a data convertion, but not in Dixer.
Do you want load 7 million rows to a Excel file? In SSIS you need Excel installed and you can't pass all these rows because excel limit (at least deal with a tedious process), with Dixer by default create a sheet an continue with the data.. Doesn't need Excel or other requeriments.
If you want to execute your package in another host, only copy the binary, no need to install anything! (Well, maybe dependences that require the Oracle Instant Client if want to work with Oracle or clidriver to work with IBM DB2).
Simplicity: You can create a complete workflow from scrath using your favorite text editor and format (TOML, YAML, JSON).
Features¶
Dixer offers a variety of features. The importans are called Jobs with can do this functions:
- Data transformation
- Data migration
- Query execution
- Mail sending
- Process task
- File operations
- FTP operations
- HTTP request
- SOAP request
- Delay
- Bucle of workflow
- Execute a group of jobs or workflow
- Compress
- Decompress
- Amazon S3 tasks
Aditional features¶
- Define variables and set it in almost all part of the project.
- Dixer supports expressions to get a custom value.
- Skip a row from migration if this match with a Rule Expression configured.
- Is possible to use derivated columns in mapping for dataflows with Expression Mapping or a variable easily.
- Execute a different workflow for each row migrated from source to destination (example: for each row send a email).
- Events Handling for execute a workflow when a Success, Completion or Error happenned in a Job
- Execute parallels Jobs
- Security: you can encrypt your package and execute with a password for execution.
- Encrypt password connections: you can encrypt your password of connections to avoid others see your passwords in package file.
- Multiple formats for your package file: You can create your package in TOML, YAML or JSON, choose your favorite!
- Ability to execute a job or complete workflow in a list of defined operative systems.
Supported sources and destinations¶
Dixer allow to migrate data from and to:
- Microsoft SQL Server*
- MySQL
- IBM DB2*
- Oracle*
- Excel file (.xlsx)
- CSV
- XML
- JSON
- Firebird SQL
- PostgreSQL
- Variables
- SQLite3*
- Ragged Right File
- Fixed Width File
- SAP HANA DB
- HTML Table (only destination)
- Markdown Table (only destination)
*Microsoft SQL Server requires TLS 1.2 (MSSQL 2008 SP4 or superior, MSSQL 2008 R2 SP3 or superior).
*IBM-DB2 requires clidriver, use this link or this to download it. In Windows is runtime need it, but others OS need it to execute the Dixer binary.
*Oracle requires Oracle Instant Client at runtime.
*SQLite3 authentication with user and password not supported. Also encrypted databases not supported.
Used libraries¶
Dixer stands on the shoulder of many great open source libraries, in lexical order: