Pentaho Data Integration (PDI)

General Information

Technologies must be operated and maintained in accordance with Federal and Department security and privacy policies and guidelines. More information on the proper use of the TRM can be found on the TRM Proper Use Tab/Section.

Website:	Go to site
Description:	Pentaho Data Integration (PDI) is an extract, transform, and load (ETL) solution that can be used to integrate, manipulate, and visualize data. PDI is used to import, transform, and export data from multiple data sources, including flat files, relational databases, Hadoop, and Not Only Structured Query Language (NoSQL) databases. Users can then display this data in a singular view. This technology has two editions: Enterprise Edition and Community Edition. The Community Edition (Kettle) is an open source version of the technology. This entry covers both editions. As of September 2017, Pentaho has merged with several companies to create a new company called Hitachi Vantara. The TRM decisions in this entry only apply to technologies and versions owned, operated, managed, patched, and version-controlled by VA. This includes technologies deployed as software installations on VMs within VA-controlled cloud environments (e.g. VA Enterprise Cloud (VAEC)). Cloud services provided by the VAEC, which are listed in the VAEC Service Catalog, and those controlled and managed by an external Cloud Service Provider (i.e. SaaS) are not in the purview of the TRM. For more information on the use of cloud services and cloud-based products within VA, including VA private clouds, please see the Enterprise Cloud Solutions Office (ECSO) Portal at: https://dvagov.sharepoint.com/sites/OITECSO This entry covers the Desktop Edition of this technology and not any mobile versions. Please note that the implementation of mobile technology applications that operate on Mobile Operating Systems must be reviewed and approved by the Mobile Technology and Endpoint Security Engineering Team: http://vaww.eie.va.gov/SysDesign/CS/MT/default.aspx
Technology/Standard Usage Requirements:	Users must ensure their use of this technology/standard is consistent with VA policies and standards, including, but not limited to, VA Handbooks 6102 and 6500; VA Directives 6004, 6513, and 6517; and National Institute of Standards and Technology (NIST) standards, including Federal Information Processing Standards (FIPS). Users must ensure sensitive data is properly protected in compliance with all VA regulations. Prior to use of this technology, users should check with their supervisor, Information Security Officer (ISO), Facility Chief Information Officer (CIO), or local Office of Information and Technology (OI&T) representative to ensure that all actions are consistent with current VA policies and procedures prior to implementation.
Section 508 Information:	This technology has been assessed by the Section 508 Office and found non-conformant. The Implementer of this technology has the responsibility to ensure the version deployed is 508-compliant. Section 508 compliance may be reviewed by the Section 508 Office and appropriate remedial action required if necessary. For additional information or assistance regarding Section 508, please contact the Section 508 Office at Section508@va.gov. Please see reference tab for more information concerning product versions.
Decision:	View Decisions

Decision Source:	TRM Mgmt Group
Decision Process:	One-VA TRM v23.11
Decision Date:	11/09/2023 at 19:23:09 UTC
Aliases:	Kettle; PDI
Introduced By:	TRM Request
Vendor Name:	Hitachi Vantara

Vendor Release Information

The Vendor Release table provides the known releases for the TRM Technology, obtained from the vendor (or from the release source).

Version	Release Date	Vendor End of Life Date	Vendor Desupport Date
3.2	05/14/2009
4.0	06/18/2010
4.1	11/30/2010
4.2	09/12/2011
4.3	05/07/2012
4.4	11/29/2012
5.0	09/12/2013
5.4	06/09/2015
6.0	10/14/2015
6.1	05/22/2016
7.0	10/31/2016
7.1	05/20/2017
8.x	11/16/2017
9.x	02/13/2020

Current Decision Matrix (11/07/2023)

Users must ensure their use of this technology/standard is consistent with VA policies and standards, including, but not limited to, VA Handbooks 6102 and 6500; VA Directives 6004, 6513, and 6517; and National Institute of Standards and Technology (NIST) standards, including Federal Information Processing Standards (FIPS). Users must ensure sensitive data is properly protected in compliance with all VA regulations. Prior to use of this technology, users should check with their supervisor, Information Security Officer (ISO), Facility Chief Information Officer (CIO), or local Office of Information and Technology (OI&T) representative to ensure that all actions are consistent with current VA policies and procedures prior to implementation.

The VA Decision Matrix displays the current and future VA IT position regarding different releases of a TRM entry. These decisions are based upon the best information available as of the most current date. The consumer of this information has the responsibility to consult the organizations responsible for the desktop, testing, and/or production environments to ensure that the target version of the technology will be supported. Any major.minor version that is not listed in the VA Decision Matrix is considered unapproved for use.

Legend:

White	Approved: The technology/standard has been approved for use.
Yellow	Approved w/Constraints: The technology/standard can be used within the specified constraints located below the decision matrix in the footnote[1] and on the General tab.
Gray	Unapproved: This technology or standard can be used only if a POA&M review is conducted and signed by the Authorizing Official Designated Representative (AODR) as designated by the Authorizing Official (AO) or designee and based upon a recommendation from the POA&M Compliance Enforcement, has been granted to the project team or organization that wishes to use the technology.
Orange	Divest: VA has decided to divest itself on the use of the technology/standard. As a result, all projects currently utilizing the technology/standard must plan to eliminate their use of the technology/standard. Additional information on when the entry is projected to become unapproved may be found on the Decision tab for the specific entry.
Black	Prohibited: The technology/standard is not (currently) permitted to be used under any circumstances.
Blue	Planning/Evaluation Constraint:The period of time this technology is currently being evaluated, reviewed, and tested in controlled environments. Use of this technology is strictly controlled and not available for use within the general population. If a customer would like to use this technology, please work with your local or Regional OI&T office and contact the appropriate evaluation office displayed in the notes below the decision matrix. The Local or Regional OI&T office should submit an inquiry to the TRM if they require further assistance or if the evaluating office is not listed in the notes below.

Release/Version Information:

VA decisions for specific versions may include a ‘.x’ wildcard, which denotes a decision that pertains to a range of multiple versions.

For example, a technology approved with a decision for 7.x would cover any version of 7.(Anything) - 7.(Anything). However, a 7.4.x decision would cover any version of 7.4.(Anything), but would not cover any version of 7.5.x or 7.6.x on the TRM.

VA decisions for specific versions may include ‘+’ symbols; which denotes that the decision for the version specified also includes versions greater than what is specified but is not to exceed or affect previous decimal places.

For example, a technology approved with a decision for 12.6.4+ would cover any version that is greater than 12.6.4, but would not exceed the .6 decimal ie: 12.6.401 is ok, 12.6.5 is ok, 12.6.9 is ok, however 12.7.0 or 13.0 is not.

<Past

CY2016

CY2017

CY2018

Future>

Release

Q1

Q2

Q3

Q4

Q1

Q2

Q3

Q4

Q1

Q2

Q3

Q4

4.4

Unapproved

5.0

Approved w/Constraints
[1]

Approved w/Constraints
[2, 3, 4, 5, 6]

Approved w/Constraints
[4, 5, 6, 7, 8]

Divest
[4, 5, 6, 7, 8]

Divest
[6, 8, 9, 10, 11, 12]

Unapproved

5.4

Approved w/Constraints
[1]

Approved w/Constraints
[2, 3, 4, 5, 6]

Approved w/Constraints
[4, 5, 6, 7, 8]

Divest
[4, 5, 6, 7, 8]

Divest
[6, 8, 9, 10, 11, 12]

Unapproved

6.0

Approved w/Constraints
[1]

Approved w/Constraints
[2, 3, 4, 5, 6]

Approved w/Constraints
[4, 5, 6, 7, 8]

Approved w/Constraints
[6, 8, 9, 10, 11, 12]

Approved w/Constraints
[6, 8, 9, 10, 11, 13]

Divest
[6, 8, 10, 11, 13, 14]

6.1

Unapproved

Approved w/Constraints
[4, 5, 6, 7, 8]

Approved w/Constraints
[6, 8, 9, 10, 11, 12]

Approved w/Constraints
[6, 8, 9, 10, 11, 13]

Divest
[6, 8, 10, 11, 13, 14]

7.1

Unapproved

Approved w/Constraints
[4, 5, 6, 7, 8]

Approved w/Constraints
[6, 8, 9, 10, 11, 12]

Approved w/Constraints
[6, 8, 9, 10, 11, 13]

Divest
[6, 8, 10, 11, 13, 14]

8.x

Unapproved

Approved w/Constraints
[6, 8, 10, 11, 13, 14]

9.x

Unapproved

Decision Constraints


[1]	Federal Information Processing Standards (FIPS 140-2) certified encryption must be used to encrypt data in transit if Personally Identifiable Information (PII) or Personal Health Information (PHI), or Veteran Affairs (VA) sensitive information is involved or additional mitigating controls must be documented in an approved System Security Plan (SSP). VA users must ensure that VA sensitive data is properly protected in accordance to VA 6500 and the Federal Information Security Management Act (FISMA).

[2]	If free trialware is utilized, the software must be purchased or removed at the end of the trial period.

[3]	In cases where the technology is used for external connections, a full Enterprise Security Change Control Board (ESCCB) review is required in accordance VA Directive 6004 , VA Directive 6517, and VA Directive 6513. The local ISO can advise on the ESCCB review process.

[4]	Per the May 5th, 2015 memorandum from the VA Chief Information Security Officer (CISO) FIPS 140-2 Validate Full Disk Encryption (FOE) for Data at Rest in Database Management Systems (DBMS) and in accordance with Federal requirements and VA policy, database management must use Federal Information Processing Standards (FIPS) 140-2 compliant encryption to protect the confidentiality and integrity of VA information at rest at the application level. If FIPS 140-2 encryption at the application level is not technically possible, FIPS 140-2 compliant full disk encryption (FOE) must be implemented on the hard drive where the DBMS resides. Appropriate access enforcement and physical security control must also be implemented. All instances of deployment using this technology should be reviewed to ensure compliance with VA Handbook 6500 and National Institute of Standards and Technology (NIST) standards. It is the responsibility of the system owner to work with the local CIO (or designee) and Information Security Officer (ISO) to ensure that a compliant DBMS technology is selected and that if needed, mitigating controls are in place and documented in a System Security Plan (SSP).

[5]	Veterans Affairs (VA) users must ensure VA sensitive data is properly protected in compliance with all VA regulations. All instances of deployment using this technology should be reviewed by the local ISO (Information Security Officer) to ensure compliance with VA Handbook 6500.

[6]	Due to National Institute of Standards and Technology (NIST) identified security vulnerabilities, extra vigilance should be applied to ensure the versions remain properly patched to mitigate known and future vulnerabilities. The local ISO can provide assistance in reviewing the NIST vulnerabilities.

[7]	If free trial ware is utilized, the software must be purchased or removed at the end of the trial period.

[8]	Users should check with their supervisor, Information Security Office (ISO) or local OIT representative for permission to download and use this software. Downloaded software must always be scanned for viruses prior to installation to prevent adware or malware. Freeware may only be downloaded directly from the primary site that the creator of the software has advertised for public download and user or development community engagement. Users should note, any attempt by the installation process to install any additional, unrelated software is not approved and the user should take the proper steps to decline those installations.

[9]	This product can be configured with a MySQL Database, which currently has TRM constraints for intranet use only due to its many known security issues. If MySQL is selected for use with this product, these factors should be considered especially when an instance of this product will be considered a Moderate or High Risk system (Refer to MySQL Database TRM entry for more details). This product can be configured with PostgreSQL which will be unapproved in Quarter 1 of 2018. Users must consider alternate databases prior to implementing this technology with PostgreSQL.

[10]	Per the May 5th, 2015 memorandum from the VA Chief Information Security Officer (CISO) FIPS 140-2 Validate Full Disk Encryption (FOE) for Data at Rest in Database Management Systems (DBMS) and in accordance with Federal requirements and VA policy, database management must use Federal Information Processing Standards (FIPS) 140-2 compliant encryption to protect the confidentiality and integrity of VA information at rest at the application level. If FIPS 140-2 encryption at the application level is not technically possible, FIPS 140-2 compliant full disk encryption (FOE) must be implemented on the hard drive where the DBMS resides. Appropriate access enforcement and physical security control must also be implemented. All instances of deployment using this technology should be reviewed to ensure compliance with VA Handbook 6500 and National Institute of Standards and Technology (NIST) standards. It is the responsibility of the system owner to work with the local CIO (or designee) and Information Security Officer (ISO) to ensure that a compliant DBMS technology is selected and that if needed, mitigating controls are in place and documented in a System Security Plan (SSP).

[11]	Veterans Affairs (VA) users must ensure VA sensitive data is properly protected in compliance with all VA regulations. All instances of deployment using this technology should be reviewed by the local ISO (Information Security Officer) to ensure compliance with VA Handbook 6500.

[12]	Due to potential information security risks, cloud based technologies may not be used without the approval of the VA Enterprise Cloud Services (ECS) Group. This body is in part responsible for ensuring organizational information, Personally Identifiable Information (PII), Protected Health Information (PHI), and VA sensitive data are not compromised. (Ref: VA Directive 6004, VA Directive 6517, VA Directive 6513 and VA Directive 6102).

[13]	Due to potential information security risks, cloud based technologies may not be used without the approval of the Enterprise Cloud Solution Office (ECSO). This body is in part responsible for ensuring organizational information, Personally Identifiable Information (PII), Protected Health Information (PHI), and VA sensitive data are not compromised. (Ref: VA Directive 6004, VA Directive 6517, VA Directive 6513 and VA Directive 6102).

[14]	If free trial ware is utilized, the technology must be purchased or removed at the end of the trial period. This technology can be configured with a My Structured Query Language (MySQL) Database, which currently has Technical Reference Model (TRM) constraints for intranet use only due to its many known security issues. If MySQL is selected for use with this technology, these factors should be considered especially when an instance of this technology will be considered a Moderate or High Risk system (Refer to MySQL Database TRM entry for more details).

Note:

At the time of writing, version 9.4 is the most current version and released 08/11/2023.

References

The following reference(s) are associated with this entry:
Type	Name	Source	Description
Vendor Provided*	Source Code		This site links to the source code for Kettle, this technology`s open source edition.
*NOTE: Vendor provided information is subject to the accuracy of the manufacturer.

Technology Components

Note: This list may not be complete. No component, listed or unlisted, may be used outside of the technology in which it is released. The usage decision for a component is found in the Decision and Decision Constraints.
Name	Description
Agile Business Intelligence (BI)	Agile Business Intelligence (BI) links Spoon`s data integration to Analyzer and its visualizations.
Carte	Carte is a web server that enables remote execution of jobs and transformations.
Data Integration Server	The Data Integration (DI) Server runs centrally stored transformations and jobs. The DI Server also hosts the DI repository and processing engine, provides a service layer for security and authentication, and allows scheduling.
Kitchen	This is a command line application for executing jobs modeled in Spoon.
Pan	Pan is a command line application for executing transformations modeled in Spoon.
Spoon	Spoon is the design interface for building jobs and transformations.

VA Categories View the VA Categorization Framework

Domain Area	Category	Customer Guidance	Category Locked By ELT Date
Information Management Technologies Data Integration	Extract, Transform, Load (ETL)

Open Source

This entry is available as an open source technology.

Operating Systems Supported by the Technology

macOS - Approved w/Constraints
Red Hat Enterprise Linux (RHEL) - Approved w/Constraints
Windows Client - Approved w/Constraints
Windows Server - Approved w/Constraints

Technologies/Standards Relationships

Runtime Dependencies:

Apache Tomcat - (required for some configurations)
Java Runtime Environment (JRE) - Oracle - (required for some configurations)
JBoss Enterprise Application Platform (EAP) - (required for some configurations)
Oracle Database - (required for some configurations)
PostgreSQL - (required for some configurations)

Comparable Technologies:

Companion Technologies/Supported Standards:

General Analysis

Adoption Benefits

At the time of writing, no National Institute of Standards and Technology (NIST) vulnerabilities had been reported and no VA Cyber Security Operations Center (CSOC) bulletins had been issued for the latest versions of this technology.
The Community edition of this technology is available free of charge via open source licensing.
This is a well-documented technology.

Adoption Risks

The Enterprise edition of this technology has a free trial period, which carries the potential for the disruption of service delivery and inaccurate financial planning.
There is no indication based on available vendor documentation that this technology is Federal Information Processing Standard (FIPS) 140-2 compliant.
This product can be configured with a PostgreSQL Database, which currently has TRM constraints limiting its use to Red Hat Enterprise Linux (RHEL) only due to its many known security issues on other platforms.
This technology can integrate with and alter database technologies.
The potential exists to store Personally Identifiable Information (PII), Protected Health Information (PHI) and/or VA Sensitive data and proper security standards must be followed in these cases.
Due to the rapid release schedule of this technology, the VA may be unable to update to the most recent patch and may require a deployment model requiring the use of specific versions.
This technology relies on JRE (Java Runtime Environment) - Oracle, which has a rapid release and update schedule.

Architectural Benefits

This technology is portable as it runs on multiple TRM-approved operating systems.
This technology uses multiple TRM-approved standards.

Inside VA

Budget and Performance

Inside the Media Room

National Observances

Special Events

VA Technical Reference Model v 24.7

General Information

Vendor Release Information

Current Decision Matrix (11/07/2023)

Decision Constraints

References

Technology Components

VA Categories View the VA Categorization Framework

Open Source

Operating Systems Supported by the Technology

Technologies/Standards Relationships

Runtime Dependencies:

Comparable Technologies:

Companion Technologies/Supported Standards:

General Analysis

Adoption Benefits

Adoption Risks

Architectural Benefits

CONNECT

VA HOME

QUICK LIST

RESOURCES

ADMINISTRATION

VA Technical Reference Model v 24.7

General Information

Vendor Release Information

Current Decision Matrix (11/07/2023)

Decision Constraints

References

Technology Components

VA Categories View the VA Categorization Framework

Open Source

Operating Systems Supported by the Technology

Technologies/Standards Relationships

Runtime Dependencies:

Comparable Technologies:

Companion Technologies/Supported Standards:

General Analysis

Adoption Benefits

Adoption Risks

Architectural Benefits

EMAIL UPDATES