IBM Power Ideas Portal
This portal is to open public enhancement requests against IBM Power Systems products, including IBM i. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).
Shape the future of IBM!
We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:
Search existing ideas
Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,
Post your ideas
Post an idea.
Get feedback from the IBM team and other customers to refine your idea.
Follow the idea through the IBM Ideas process.
Specific links you will want to bookmark for future use
Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.
IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.
ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.
IBM believes this request is properly supported with the current RUNSQLSTM implementation for reading integrated file system files.
RUNSQLSTM handles any file within the IFS as long as the stream file is marked with the correct CCSID. If the file is marked as CCSID 1208 and contains a UTF-8 BOM, the BOM is recognized and ignored. When the file is tagged with a different CCSID, there is no attempt to recognize and handle a UTF-8 BOM and the extra bytes will cause a failure as described in this RFE.
The UTF-8 BOM is required for proper support on Windows since Windows files do not have an attribute indicating the CCSID of the data within the file. However, IBM i files have an attribute indicating the CCSID of the data within the file, so the UTF-8 BOM is not used for this purpose. The IBM i CCSID conversion code correctly handles the BOM, but that code only looks for a BOM if the file is marked with a Unicode CCSID.
The reason that things are not working as expected is likely due an incorrect CCSID being set when the file is created or moved to the IBM i IFS file system. When a file is transferred from Windows to IFS, when the IFS file is created, a CCSID gets set on the IFS file. The default is to create the file with one of the ASCII CCSIDs 1252, 437, or 819. If the file is created with one of these CCSIDs and contains a UTF-8 BOM, the file is not tagged properly. As such, it may have unexpected results with any application on the system. For example, WRKLNK with the file shows the BOM as three characters at the start of the data.
In order to have the file properly interpreted on the IBM i, it needs to be marked as CCSID 1208. Here are two possible commands that can be used to change the CCSID attribute of an existing file:
CHGATR OBJ('/path/to/myfile.txt') ATR(*CCSID) VALUE(1208)
STRQSH CMD('setccsid 1208 /path/to/myfile.txt')
These commands do not change any of the data within the file, but only change the CCSID attribute of the file. There are other methods that can be used as well.
Once the file has been properly tagged with the correct CCSID, RUNSQLSTM (and any other applications) will be able to properly read the contents of the file. For example, WRKLNK shows the contents of the file without the three extra characters from the BOM.
Due to processing by IBM, this request was reassigned to have the following updated attributes:
Brand - Servers and Systems Software
Product family - Power Systems
Product - IBM i
Component - Db2 for i
Operating system - IBM i
Source - None
For recording keeping, the previous attributes were:
Brand - Servers and Systems Software
Product family - Power Systems
Product - IBM i
Component - Languages - CL (Control Language)
Operating system - IBM i
Source - None
From your example I'm speculating that what you're saying is the bite order Mark or BOM. Believe there's an option to turn that off. Our iQuery product handles UTF8 and the BOM so I don't recall where that ACS option is...