Table of Contents
[ Up To Index ]
Work With Merge Engines
OverView
Merge engines are processes that watch outqueues. When a spool file arrives in the outqueue or is released in it the engine proceeds through the specified tests looking for a match with the spool file. If a match is found the engine processes the spool file with the specified form, check or form set and moves the original spool file to the archive outqueue.
The engine is a Data Queue Engine if a data queue is given. Otherwise the engine is a Polling Engine. Data Queue Engines block until a spool file arrives or is released in the outqueue and therefore are more economical with system resources than Polling Engines, which periodically poll the outqueue.
Engines are batch jobs that run in batch subsystems. The iDocs Utility Menu has an option to create the FMGBATCH subsystem if the user does not wish to run engines in QBATCH.
Engine List Display
Option 4 from menu IDOCS displays the list of all defined engines:
4/29/08 iDocs Form Suite FM0450C1
11:27:42 Engine Definition Maintenance QSECOFR
Logging Status: *ON Hold Archive Status: *ON
1=Select 3=Copy 4=Delete 5=View 7=Rename 8=Start 9=End 10=Thread 11=Group
18=Start (FMC0459)
Opt Engine Description Status
DCYTEST DCY Test *NONE
POLLING Test Engine *NONE
QUICKTEST QuickTest *NONE
TESTCDA Test Engine cda *NONE
TESTENG Test Engine *NONE
MYENGINE My Engine *NONE
Bottom
F3=Exit F6=Add F7=Toggle Logging F8=Toggle Hold Status F21=Command
Logging and Archive Status
The status display shows the current status of the engine.
| Logging Status: | *ON - Engine logging is active |
| Hold Archive Status: | *ON - Archived entries will be placed on hold status |
Note:
Logging is a debugging tool. *ON enables engine logging and each engine process will record an entry in a data base file named FRMLOGF. Normally this value can be set to *OFF.
Hold Archive *ON places the resulting FMGLASER entries on hold (HLD) in the designated Archive output queue. An *OFF designation leaves the entries with a ready (RDY) status.
Engine Status
The status display shows the current status of the engine.
| *NONE - the engine is not running | |
| *ACTIVE - the engine is running in a subsystem | |
| *JOBQ - the engine is waiting in a jobqueue to enter a subsystem to run |
Note: For the display to detect a running engine, the job name in the subsystem must be the engine name. This setting is automatically made when launching the engine from this display using option 8. If you are manually starting an engine with the Submit Job command (SBMJOB) you must set this parameter:
SBMJOB CMD(CALL PGM(FMC0462) PARM('MYENGINE')) JOB(MYENGINE)
The above command starts a data queue engine named MYENGINE, specifying MYENGINE as the job name. MYENGINE will now show active in the engine display screen.
There is the same issue with automatic start/stop of engines.
Options
Option 8 Start Engine
Select this option to start the engine into a subsystem as a batch job. Specify the appropriate jobqueue at the prompt. Use QS36EVOKE if the subsystem FMGBATCH is not set up, otherwise use FMGJOBQ. Bear in mind that this subsystem and jobqueue should be multi-threaded. The QBATCH jobqueue ships from IBM as single threaded. The subsystem FMGBATCH and its jobqueue FMGJOBQ ship configured to support 2 concurrent engines. Any properly configured jobqueue that routes into any properly configured subsystem may be specified.
Option 9 End Engine
Select this option to end an engine. A datqueue engine will stop at the next point of inactivity. A polling engine will stop at the time of its next polling cycle, i.e. at the end of its wait period.
Option 10 Thread Engine
Select this option to start the data queue engine as a Thread Engine. A Thread engine is used when large print volumes are anticipated. It will prevent the 9999 spooled file limit for a data queue, where a job will error out when it has reached it max and need to be restarted. Each spool file job is batched instead of being run interactively. The batch jobs are sent to jobq QTXTSRCH as the default. The data area FMG/DSJOBQMON is used to store the default job queue used and can be changed if desired.
Note: If the Thread Engine is used, an archive output queue must be specified.
Option 18=Start (FMC0459)
This is fundamentally identical to selecting with Option 8, but provides the ability to retain the originating spool file user attributes for the resulting output.
Add / Maintain Merge Engine
To work with merge engines, go to menu IDOCS and take
4. Work with Engines
Use F6 to create a new engine or select an existing engine (not running) with option 1.
3/15/14 iDocs Form Suite FM0450C3
07:37:01 Engine Definition Maintenance DYOKANA
Engine Name. . . . . . : IDOCSENG Description. . : iDocs Engine
Archive Out Queue. . . : ARCHIVE Archive Library: IDOCS661 F4=List
Data Queue . . . . . . : IDOCSDTAQ Dataque Library: IDOCS661
Engine Status. . . . . : Custom Command:
Engine Type . . . . . : *SERIAL Test Type . . .: *FIRST *FIRST, *ALL
Originating Spool Spool Form Resulting Resulting
Prc Out Queue Library Attribute Attribute App SpoolFile Out Queue
Grp F4=List F4=List Value F4=List Name F4=List
IN QUSRSYS *FORM INVOICE INVOICE OUT
More...
F3=Exit F4=Prompt F7=Server View F8=Aggregation Settings F12=Previous F21=CMD
5250 MW 075/012
Header Information
- Engine Name: Name of the engine. Must be unique. Do not use embedded blanks.
- Description: A description of the engine.
- Archive Out Queue: Outqueue in which to place the input *SCS spool file after processing. If *NONE is specified the spool file will be deleted after processing. iDocs does not maintain the archive outqueue, so the administrator must periodically delete spool files from the outqueue.
- Archive Library: Library containing archive outqueue.
- Data Queue: Enter the name of a dataqueue to use with this engine or leave blank to run as a Polling Engine. The interval between pollings is specified in seconds in the data area DSWAIT. Engines should not share dataqueues.
- Dataque Library: Library containing the dataqueue.
- Engine Status: Current status of the engine.
- Custom Command: Command to invoke a custom engine.
- Engine Type: *SERIAL for normal processing and *AGGREGATE to perform aggregation.
- Test Type:: *FIRST to process just the first target matched or *ALL to process every matching target.
Detail Information
These are tests applied to each spool file arriving in the outqueue. A spool file is processed once with the form, check or form set specified for the first test that returns true.
- Prc Grp: For future use.
- Originating Outqueue / Library: Specify the outqueue to monitor.
| Originating Outqueue Considerations |
|---|
| Do not monitor outqueues attached to a printer or monitored by another engine as which entity will process the spool file is unpredictable. Use F4 to select the outqueue from a list. |
| Do not monitor an outqueue with the same name as any engine as the engine uses an outqueue by its own name in FMG as a work outqueue. |
- Spool Attribute: Attribute of the spool file to test. Use F4 to select the spool attribute from a list of supported values.
- Spool Attribute Value: Value to use in the spool attribute test.
- Form App: Form, check or form set to use in processing the spool file. Use F4 to select the form, check or form set from a list of these objects.
- Resulting SpoolFile Name: The output PCL spooled file name from iDocs Engines is named “FMGLASER” reflective of its contents. You can specify a resulting spool file name here. A new enhancement in iDOCS 8 (09/19/23) allows administrators to configure the iDocs Engine to use the originating spool file name as the output PCL spool file name's name. To configure the iDocs Engine to use the originating spool file’s name as the PCL’s spool file name use *SPLFNAME or *FILE as values in the Resulting SpoolFile Name field on the iDocs Engine line. See the below example where the first line in the iDocs Engine configuration below will use FMGLASER as the PCL File Name in the output. The second line in the iDocs Engine configuration will use the originating spool file’s name as the PCL File Name in the output.
- Resulting Out Queue: Outqueue to place the resulting *USERASCII spool file. This is the electronic document. Use F4 to specify the outqueue from a list.
| Resulting Outqueue Considerations |
|---|
| Specify *FORMDEF to use the destination outqueues from the form definition. |
| Any destination outqueue specified in the form will override this. |
| DO NOT specify an outqueue in library FMG of the same name as the engine. |
Server View Parameters
Originating Spool Spool Form Resulting Resulting
Prc Out Queue Library Attribute Attribute App SpoolFile Out Queue
Grp F4=List F4=List Value F4=List Name F4=List
PRT02 QUSRSYS *FORM *STD DEMOCHK *FORMDEF
Archive Queue: Library: Second File:
Archive Queue: Library: Second File:
From the main screen you may press F7 to specify these additional parameters:
- Archive Queue: The archive outqueue specifically for this spool file test.
- Library: The library containing the archive outqueue for this spool file test.
- Second File: If a second file is specified, when the engine line is triggered through a successful test, the engine will append the contents of the named second spool file to the first spool file (which triggered the engine test) before performing the merge.
Aggregation options
In iDocs 6.5 the Aggregation Engine feature was added. This option allows you to combine multiple spool files into a single pdf to be emailed.
The aggregating engine operates by accumulating output until the trigger, as defined on the screen above, returns true. At that time it outputs the accumulated document ( as one document ) and clears the document cache. This feature has its most common use in creating one unified attachment for email delivery. If there are multiple documents that need be attached to an email, then this feature can amalgamate them into one attachment.
4/21/20 iDocs Form Suite FM0450C3
14:10:15 Engine Definition Maintenance KKRAMER
Engine Name. . . . . . : SIGMAENG Description. . : SIGMA Test Engine
Archive Out Queue. . . : SIGMASPLF Archive Library: QGPL
Data Queue . . . . . . : SIGMAENGDQ Dataque Library: QUSRSYS
Engine Status. . . . . : *ACTIVE Custom Command:
Engine Type . . . . . : *AGGREGATE Test Type . . .: *FIRST *FIRST, *ALL
Originating Spool Spool Form Resulting Resulting
Prc Out Queue Library Attribute Attribute App SpoolFile Out Queue
Grp Value Name
SIGMAIN QUSRSYS *FORM ORDER_ACK SGACKNW IN
SIGMAIN QUSRSYS *USRDTA PO610A SGPOFORM IN
SIGMAIN QUSRSYS *FORM ORD_SHIPTX SGSO IN
More...
F3=Exit F7=Server View F8=Aggregation Settings F12=Previous F21=CMD
By creating the engine as Engine Type Aggregate, you can utilize option F8 to enter the Aggregate Settings.
FM0450W8
Aggregate Setting
Select Form to Combine/Aggregate . . . . . *ALL
User program to check aggregate condition. TTRAN USERPGM
Sweep the outputs when
AND/OR Field Op Value (Characters)
More...
F4=Prompt F5=Refresh F10=Save F12=Previous F13=PGM Template
In the Aggregate Setting screen shown above, the user can either specify a custom program (IDOCS 6.95) to handle the business logic of Aggregation or define “Sweep the outputs when”. The custom program will override the definition in “Sweep the outputs when”.
F13 will assist the user in creating a program along with the parameters that the iDocs engine will utilize when calling the the custom program. A sample CL program is shown below.
/*¹iDocs sample user program for Aggregate engine· +
¹Once this program is specified in iDocs aggregate engine, +
¹iDocs will call it to check if the aggregate criteria is matched or not, +
¹if TRUE then &RETURN is 'Y' else &RETURN is blank. +
¹ +
¹Input parameters: +
¹ - &SPLNAME : SCS spool file name which is being processed by iDocs engine +
¹ - &SPLNBR : SCS spool file number +
¹ - &JOBNAM : SCS spool file job name +
¹ - &JOBUSR : SCS spool file job user +
¹ - &JOBNBR : SCS spool file job number +
¹ +
¹Output parameters: +
¹ - &RETURN : If this parameter is Y then iDocs will complete the group +
¹ and aggregate all PCLs in group into 1 PCL +
*/
PGM PARM(&SPLNAME &SPLNBR &JOBNAME &JOBUSER +
&JOBNBR &RETURN)
DCL VAR(&SPLNAME) TYPE(*CHAR) LEN(10)
DCL VAR(&SPLNBR) TYPE(*DEC) LEN(6 0)
DCL VAR(&JOBNAME) TYPE(*CHAR) LEN(10)
DCL VAR(&JOBUSER) TYPE(*CHAR) LEN(10)
DCL VAR(&JOBNBR) TYPE(*CHAR) LEN(6)
DCL VAR(&RETURN) TYPE(*CHAR) LEN(1)
/*¹If &RETURN is 'Y' then it's end of the spool files group, +
¹ and iDocs will aggregate the PCL files into 1 PCL·*/
ENDPGM
Daisy Chaining Engines
One of the architectural advantages of the iDocs Suite is that a spool file can be processed by multiple engines serially. This enables functionality impossible within one form, formset or engine.
The key mechanism of daisy chained engines is the engine archive outqueue. The normal use of the archive outqueue is to store the original spool file for re-use. However, it can be used to hand off the original spool file to another engine by specifying as the archive outqueue an outqueue that is monitored as an originating outqueue by another engine.
------------- -------------- ------------- ------------ ---------------
/ engine 1 / -->>> / engine / -->>>> / engine 1 / -->>>> / engine 2 / -->>> / engine 2 /
/ orig outq / / 1 / / archive \ / / / / archive outq /
------------ -------------- / engine 2 / ------------ ----------------
| / orig outq/ |
\ -------------- \
\ \
----------- -----------
/ normal / / normal /
/ output / / output /
------------ ------------
The engine setups would look something like this:
Engine 1 Setup:
6/08/10 iDocs Form Suite FM0450C3
11:27:27 Engine Definition Maintenance DYOKANA
Engine Name. . . . . . : ENGINE1 Description. . : First Engine
Archive Out Queue. . . : ARCHIVE1 Archive Library: FMG F4=List
Data Queue . . . . . . : DTAQ1 Dataque Library: FMG
Engine Status. . . . . : Custom Command:
Remove Over Printing: N
Originating Spool Spool Form Resulting Resulting
Prc Out Queue Library Attribute Attribute App SpoolFile Out Queue
Grp F4=List F4=List Value F4=List Name F4=List
ORIGOUTQ1 FMG *FORM *STD FORMAPP1 NORMALOUT1
Engine 2 Setup:
6/08/10 iDocs Form Suite FM0450C3
11:27:27 Engine Definition Maintenance DYOKANA
Engine Name. . . . . . : ENGINE2 Description. . : Second Engine
Archive Out Queue. . . : ARCHIVE2 Archive Library: FMG F4=List
Data Queue . . . . . . : DTAQ2 Dataque Library: FMG
Engine Status. . . . . : Custom Command:
Remove Over Printing: N
Originating Spool Spool Form Resulting Resulting
Prc Out Queue Library Attribute Attribute App SpoolFile Out Queue
Grp F4=List F4=List Value F4=List Name F4=List
ARCHIVE1 FMG *FORM *STD FORMAPP2 NORMALOUT2
The normal use of daisy chaining is to automate complex functionality. Some possible uses include:
- Process a document to print and then archive to networked directory using the iPDF Monitor.
- Process a document to email, then to fax, then to print where users get more than one output type.
(i.e. a user can get both an email and a fax )
- Archive a spool file using iView then email using the iMail Application Server.
Configuring Automatic Engine Start / Stop
Engines can be configured to automatically start and stop two ways.
Starting Engines at IPL
Attention: This change should be done by a properly qualified programmer. A mistake in the startup program can render your system dysfunctional.
Presuming that the FMGBATCH subsystem has been installed and configured on your system, alter the system startup program ( system value QSTRUPPGM ) to execute the following:
ADDLIBLE FMG *FIRST
STRSBS FMGBATCH
SBMJOB CMD(CALL PGM(FMC0462) PARM(ENGNAME)) JOB(ENGNAME) JOBD(FMGJOBD) JOBQ(FMGJOBQ) USER(FMGUSER)
Discussion:
- The FMGBATCH subsystem can be created from menu FMGUTILITY, option 1. The default configuration is limited and the configuration should be reviewed by a qualified system administrator.
- Never call the iDocs engine from the startup program, instead submit it with SBMJOB. If you call it the startup program will never progress beyond the call statement and thus never complete. Additionally CALLing the engine will cause it to run under the user profile running the startup program (user QSYS) which causes problems. iDocs engines will run in any properly configured batch subsystem.
- Program FMC0462 is a data queue engine; use FMC0460 for a polling engine. (FMR0471 ends either one).
- The one parameter to the FMC0462 or FMC0460 call should be the name of the engine in upper case.
- The job name should be the engine name in upper case.
- You can control library list and execution environment by specifying the job description as FMGJOBD, or another job description of your creation. FMGJOBD ships with a default configuration and should be reviewed / changed by a qualified system administrator.
- Always specify a user in the SBMJOB command so that user QSYS does not run the engine. The user needs *SPLCTL and *JOBCTL authorities, as well as an entry in the System Directory so that it has access to folder FORMFMG. It will also require access to library FMG as well as folder FORMFMG specifically.
Configuring the Engines to Start and Stop
You may create generic Job Schedule entries to start and stop engines from menu FMGUTILITY, option 2. See Schedule an Engine to Start/stop .
The easiest way to create entries is with the Utility menu. This option will generate JOBSCDE entries. You can them view the entries (WRKJOBSCDE) and modify them as needed.
Examples: To start and end a Data Queue engine named ENGINE1 in library IDOCS:
To start: CALL PGM(IDOCS/FMC0462) PARM('ENGINE1')
To end: CALL PGM(IDOCS/FMR0471) PARM('ENGINE1')
Proper Target Outqueue Configuration
Essentially, iDocs is a pcl 5e and pcl 5c printer driver for the IBM i. Its output is printer ready pcl. As such its output does not need translating, and in fact such translation may corrupt the output.
There are two considerations:
- The destination outqueue on the IBM i should specify TRANSFORM(*NO) so that the output is not transformed as it leaves the IBM i.
- The destination outqueue on the IBM i should specify as the target queue on the target printer a queue that does not transform the input so that the target printer does not transform the stream. For Hewlitt Packard printers, that queue is named RAW. So the IBM i outqueue should contain this configuration parameter: RMTPRTQ('RAW').
Engine User Profile Issues
Profile Settings
The profile running the engine ( IDOCS in this section ) should have the following attributes set in the user profile:
- USRCLS(*SYSOPR):User class *SYSOPR
- SPCAUT(*JOBCTL *SPLCTL): Special authorities: job control, spool control
- MAXSTG(*NOMAX): No ( or very large ) maximum storage limit for profile
- JOBD(FMGJOBD): Job description should be FMGJOBD, which has a limited and correct library list specified
- OWNER(*USRPRF): New objects should be owned by the profile
- HOMEDIR('/home/idocs'): Specify a home directory for the profile; make certain it exists and that the profile has full authority over it.
The user profile should have access to the iDocs library:
GRTOBJAUT OBJ(iDocs library) OBJTYPE(*LIB) USER(IDOCS) AUT(*ALL)
GRTOBJAUT OBJ(iDocs library /*ALL) OBJTYPE(*ALL) USER(IDOCS) AUT(*ALL)
You may use the below command to permanently grant the required permissions for iDocs:
CHGAUT OBJ('/qsys.lib/iDocs.library/*') USER(*PUBLIC) DTAAUT(*RWX) OBJAUT(*ALL)
If there are no object authority issues with iDocs you might want to disable adopted authority on the programs:
CHGPGM PGM(iDocs library /*ALL) USRPRF(*USER) USEADPAUT(*NO)
Job Description Settings
FMGJOBD Job Description has these settings:
- JOBQ(QGPL/QS36EVOKE): Uses multithreaded jobqueue by default.
- INLLIBL(FMG QGPL QTEMP): FMG at the top of a simple library list.
- JOBMSGQFL(*WRAP): Wrap a full message queue.
Directory Entry
This user profile should also have an entry in the system directory; you can use the command ADDDIRE to add the entry:
ADDDIRE USRID(IDOCS LCLSYSTM) USRD('iDocs user profile') USER(IDOCS) SYSNAME(*LCL)
Engine Troubleshooting Q & A
Q. When I submit the engine it prompts me for the job queue. What is this?
A. A job queue is an IBM i object that contains entries for jobs that are waiting to be processed by the system.
Q. When I submit the engine it runs normally, but afterwards no one else can process their jobs submitted to batch?
A. In this case the engine has been submitted to a single thread job queue that can only process a single job at a time. The solution to this bottleneck is to submit the engine to a multi thread job queue. Create one or use a pre-configured multi thread job queue such as QS36EVOKE.
Q. Polling Engine starts for 60 seconds and ends normally without processing any entries?
A. Check each originating output queue and insure that they all still exist on the system.
Q.The engine runs and the original spool fie is successfully copied to the archive out queue, yet the merged spool file disappears.
A. The writer may not be responding correctly. Follow these steps to fix the writer: end the writer, vary off the device, vary on the device with a reset, restart the writer. If this doesn't resolve the issue, verify that the printer is HP4 compatible.
Q. After the engine is submitted, it ends several days later in error?
A. There could be numerous reasons why the engine has ended in error after successfully running without interruption. Most common is that the engine has been running for many weeks accumulating system resources and job structures. It is recommended that an engine be stopped and restarted weekly. Other specific causes:
- A lock on the engine record has occurred from a backup routine. Its best to make a single backup of the iDocs library whenever the definitions have been added or altered and set it aside, then remove the iDocs library from the backup definition.
- An engine will end abnormally if the job's message queue has filled up. This could be the case if the engine has been running for weeks. To remedy this type of abnormal end change the system value QJOBMSGQFL value to *WRAP. This will prevent the job queue from filling up. CHGSYSVAL SYSVAL(QJOBMSGQFL) VALUE(*WRAP).
- If the engine has been running for weeks, it could end abnormally if it exceeds the maximum printer output files that can be created per job. Support was added to V5R1M0 to allow a job to create up to 999,999 spooled files. To remedy this type of abnormal end change the IBM i system value QMAXSPLF value from the default value of 9999 to the maximum of 999999. CHGSYSVAL SYSVAL(QMAXSPLF) VALUE(999999).
- If the engine is processing a large volume of spool files there could be an abend due to 'QSPOOL job not closing'. Keeping spooled files with jobs allows job commands such as Work with Submitted Jobs (WRKSBMJOB) to work with the spooled files even after the job has ended. Detaching spooled files from jobs reduces the use of system resources by allowing job structures to be recycled when the job ends. The default for QSPLFACN is *KEEP. CHGSYSVAL SYSVAL(QSPLFACN) VALUE(*DETACH): When the job ends, the spooled files are detached from the job and the job is removed from the system.
Note: You must have *ALLOBJ and *SECADM special authorities to change system values. A change to the system value takes effect immediately; therefore, there is no need to IPL the system after making the change.
[ Up To Index ]