Skip to Main Content
IBM Power Ideas Portal


This portal is to open public enhancement requests against IBM Power Systems products, including IBM i. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:

Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,

Post your ideas
  1. Post an idea.

  2. Get feedback from the IBM team and other customers to refine your idea.

  3. Follow the idea through the IBM Ideas process.


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

Status Delivered
Workspace IBM i
Categories Db2 for i
Created by Guest
Created on Feb 20, 2019

Retain values in IBM i SQL Table Audit columns when copying rows using SQL INSERT or UPDATE

In V7R3M0, the option of new audit column types were added to tables. These are often, though not exclusively, being implemented for use in temporal tables.
An example of the audit column definitions in a CREATE TABLE for a non-temporal table statement:

CHG_MODE FOR COLUMN CHG_TYPE CHAR(1) CCSID 37
NOT NULL GENERATED ALWAYS AS ( DATA CHANGE OPERATION ),
CHG_CURRUSER FOR COLUMN CHG_USER VARCHAR(18) CCSID 37
NOT NULL GENERATED ALWAYS AS ( USER ) ,

An example of the audit column definitions in a CREATE TABLE for a temporal table statement:

START_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW BEGIN,
END_TS TIMESTAMP(12) NOT NULL GENERATED ALWAYS AS ROW END,
TS_ID TIMESTAMP(12) GENERATED ALWAYS AS TRANSACTION START ID,

Columns using the GENERATED ALWAYS clause had existed prior to V7R3M0, for identity, rowid and row change timestamp.
If performing an INSERT/SELECT, OVERRIDING SYSTEM VALUE could be used to retain the original values for identity, rowed and row change timestamp, but this isn't possible for the new columns.

However, it is very important for us and our customers that we can retain the information when populating test environments or copying the existing records of a modified table to the new version of a table.

I realize that it is possible to retain the values by using CPYF, but we need to copy the data using INSERT/SELECT or UPDATE/SELECT due to the additional functionality found in SQL that isn't possible with CPYF.


Use Case:

In our case as a change and deployment management product, we generally don't have access to the end-customer's SQL script for creating or replacing a table, particularly in production environments where only the *FILE object is permitted to be deployed from their dev/test environments. This object is then replicated to n libraries in production.

During the replication, if a version of the table already exists, it is moved to a temporary backup library and the new format of the object is moved to the target library.
The columns in the old and new versions of the file are then inspected in order to build the INSERT statement in order to map/transform the existing data from the prior format of the file to the new format of the file.

We perform an SQL INSERT rather than a CPYF for 4 reasons:
1. To automatically handle type changes, such as from numeric to alphanumeric or vice-versa.
2. To allow customer-defined transformation of specific column data during the upgrade to the new version of the file
3. To perform test environment data replication across partitions using DRDA with automatic transformation/encryption of customer-identifying data during the INSERT so that such information is at no time resident on the test systems.
4. To allow for only a couple of seconds of downtime by:
a. ensure journaling is active on the live table or start journaling when not
b. use the INSERT to bring over the n million existing records
c. use INSERT, UPDATE or DELETE to map the transactions recorded in the journal since the initial insert occurred
d. build the indexes/views over the new format of the table
e. lock the original file and then perform the move using MOVOBJ. Since a MOVOBJ occurs, hours of downtime are reduced to a few seconds, but this wouldn't work with all of the column data manipulation using CPYF.


Idea priority High
  • Guest
    Reply
    |
    Dec 1, 2022
    This enhancement was delivered in December 2022 with IBM i 7.5 PTF Group level 3 and IBM i 7.4 PTF Group level 23.

    Modifying audit column values can be performed with the use of the new REPLICATION_OVERRIDE global variable.

    This global variable is introduced on this page: https://www.ibm.com/support/pages/node/6828143
    It is described in IBM Documentation here: https://www.ibm.com/docs/en/i/7.5?topic=variables-replication-override
    and here: https://www.ibm.com/docs/en/i/7.5?topic=language-overriding-system-generation-column-values


    Db2 for i development team
    IBM Power Systems Development
  • Guest
    Reply
    |
    Jan 12, 2021

    IBM will use this request as input to planning but no commitment is made or implied. This request will be updated in the future if IBM implements it.

  • Guest
    Reply
    |
    Dec 17, 2019

    The CAAC has reviewed this requirement and recommends that IBM view this as a medium priority requirement that should be addressed. There are work-arounds but they are a bit costly to implement.

    Background: The COMMON Americas Advisory Council (CAAC) members have a broad range of experience in working with small and medium-sized IBM i customers. CAAC has a key role in working with IBM i development to help assess the value and impact of individual RFEs on the broader IBM i community, and has therefore reviewed your RFE.

    For more information about CAAC, see www.common.org/caac

    For more details about CAAC's role with RFEs, see http://www.ibmsystemsmag.com/Blogs/i-Can/May-2017/COMMON-Americas-Advisory-Council-%28CAAC%29-and-RFEs/

    Nancy Uthke-Schmucki - CAAC Program Manager