imcl and manageProfile: some examples to work with IBM products from linux command line

Working on linux servers could "force" System Engineers to work with command line for several reasons:

• automation (eg Ansible playbooks)
• because it's quicker
• security policy
• GUI not available

and many more..

I thinked to blog some examples of this command because could be useful to be launched from shell or cold be integrated inside script/playbooks.

Installation Manager , has a dedicated shell utility available inside

/opt/IBM/InstallationManager/eclipse/tools/


IBM Installation manager setup and update from public repo

setup with logfile

./installc -log log_file -acceptLicense

update to the last release from IBM public repo

./imcl install com.ibm.cic.agent

Updated to com.ibm.cic.agent_1.9.1000.20191001_1228 in the /opt/IBM/InstallationManager/eclipse directory.

List Available package

imcl sintax is based on product name and protuct level included in the package.
Use the following command to extract this infos from repository in zip format or from repository.xml included ::

/imcl listAvailablePackages -repositories WAS_V9.0_SUP_HTTPSVR.zip

com.ibm.websphere.IHS.v90_9.0.0.20160526_1854

in this case IBM   IHS 9.0 FP0

Combined setup

In case of WAS 9,x components (WASND,IHS;IHS_PLG) its required a combined setup because the component needed must be installed together the Java JDK.

the following example you could setup IHS 9.0 with JDK 8.0

./imcl install com.ibm.websphere.IHS.v90_9.0.0.20160526_1854 com.ibm.java.jdk.v8_8.0.3000.20160526_1317

 -repositories WAS_V9.0_SUP_HTTPSVR.zip,SDKJAVA_TM_TE_V8.0_LINUX.zip -acceptLicense

Installed com.ibm.websphere.IHS.v90_9.0.0.20160526_1854 to the /opt/IBM/HTTPServer directory.

Installed com.ibm.java.jdk.v8_8.0.3000.20160526_1317 to the /opt/IBM/HTTPServer directory.

Rollback

Installation manager keep track of the sw version installed and permit to roolback to previous version with a single command

./imcl listInstalledPackages -rollbackVersions

com.ibm.websphere.IHS.v90_9.0.11.20190312_2048 : 9.0.9.20180906_1004,9.0.6.20171205_1311,9.0.5.20170918_1844,9.0.0.20160526_1854

the output of this command say we have 

IHS 9 FP11, that coudl be rolled back to FP9,FP6, FP5, FP0

the following comamnd rollback IHS from FP11 to Fp9

./imcl rollback com.ibm.websphere.IHS.v90_9.0.11.20190312_2048 9.0.9.20180906_1004

Fixpack setup

The following example will upgrade WebSphere Plugin to FP10 from local repo

./imcl install com.ibm.websphere.PLG.v90_9.0.10.20181119_1807 

-repositories /data/fp10/IHS/repository.xml -acceptLicense

Updated to com.ibm.websphere.PLG.v90_9.0.10.20181119_1807 in the /opt/IBM/WebSphere/Plugins directory.

WebSphere Cell

DMGR

from WAS bin directory  use the following example to create  a cell with a DMGR profile

./manageprofiles.sh -create 

-profileName WasND 

-profilePath /opt/IBM/WebSphere/AppServer/profiles/WasND 

-templatePath /opt/IBM/WebSphere/AppServer/profileTemplates/management 

-serverType DEPLOYMENT_MANAGER 

-cellName cellName -nodeName NodeName -hostName FQDN 

-enableAdminSecurity true -adminUserName was9admin -adminPassword Password

Custom AppServer profile creation and federation

from WAS bin directory create an appserver profile

./manageprofiles.sh -create -defaultPorts -profileName AppSrv01 

-profilePath /opt/IBM/WebSphere/AppServer/profiles/AppSrv01 

-templatePath /opt/IBM/WebSphere/AppServer/profileTemplates/managed 

-hostName FQDN -nodeName nodeName -federateLater true

at this point DMGR should be tourned on and reachable and with following command  (launched from AppServer bin)  proceed with the federation

./addNode.sh dmgrFQDN 8879 -user was9admin -password Passw0rd

 Ricordatevi ovviamente di valorizzare in modo corretto i vari, cellName,hosName,nodeName e adminPassword !

All commands in this blogpost could be used on linux and windows with some minor change. 
This examples are showed with minimal options, but you could add a tons of options to imcl and manageProfile to specify all the details  needed.

 I invite you to read more details on official docs:
  

• here for imcl
• here for manageProfile

Tips & Tricks about WebSphere Application Server upgrade from 8.5.0.2 to 8.5.5 FP13 on Windows Server

During last weeks because of the JAVA & EOL I had to upgrade a WAS 8.5.0.x to 8.5.5. FP13 to be able to get latest fixes and JVM version.

First tip is you need the  refresh pack to upgrade to 8.5.5 and this RP could be installed with IBM Installation manager like every others FP .

Before to start  the upgrade if you are working on Windows server 2008 , you will need to stop WebSphere, put IBM WAS Services on manual and restart the server to avoid  possible DLL locking.

If you like to read more on this follow  this technote to get any others deatails on this (including also java.exe lock).

Fp13 is also the lastest Fp available for Windows 2008R2 because the upcoming FP14 (right now planned for august), will not support this OS (and others.. ) anymore.

WebSphere Application server: how reset a password from file registry using wsadmin

Sometimes it happens.. I had to login to a WebSpere DMGR but the password of the local administrator wasn't working as expected and there wasn't LDAP configured.

One of the easy way you have to get back your Cell is reset the password of the local admin using wsadmin

• open a command promp on  dmgr profile bin
• wsadmin -conntype NONE
• $AdminTask changeFileRegistryAccountPassword {-userId waslocal -password ab12c34de}
• $AdminTask change

Restart the JVM and you will be able to login with your waslocal userId and ab12c34de password. After this task perform a syncNode on every Node to synchronize this change.

If you need more details please read this wiki page.

WebSphere Portal how to: Recover a WPSDB on derby with SQL status XJ040

Some days ago I've received a new ticket from one of my customers with a WebSphere portal unable to start due some issue on WPSDB.

This was a dev environment so WPSDB was resident on local derby instance

inside  SystemOut.log this was the relevant error during Portal start

Failed to start database 'C:/IBM/WebSphere/wp_profile/PortalServer/derby/wpsdb' with class loader com.ibm.ws.bootstrap.ExtClassLoader@5bfa5bfa, see the next exception for details.DSRA0010E: Stato SQL = XJ040, Codice errore = 40.000
Failed to start database 'C:/IBM/WebSphere/wp_profile/PortalServer/derby/wpsdb' with class loader com.ibm.ws.bootstrap.ExtClassLoader@5bfa5bfa, see the next exception for details.
Recovery failed unexpected problem log record is Not first but transaction is not in transaction table : 40980828.
ERROR XSLAO: Recovery failed unexpected problem log record is Not first but transaction is not in transaction table : 40980828.


Inside websphere setup there are some derby libraries available under

WASHOME\derby\lib

so to verify the WPSDB status we have copied the WPSDB folder in a temporary directory than we have started the troubleshooting:

java -cp "derby.jar;derbytools.jar" org.apache.derby.tools.ij

this prompt was the response

ij>

here we tried to connect to our copy

ij> connect 'jdbc:derby:C:/temp/wpsdb';

the connections worked, so we checked the table status with the following command

SELECT schemaname || '.' || tablename as TableName,SYSCS_UTIL.SYSCS_CHECK_TABLE(schemaname, tablename) AS OK FROM sys.sysschemas s, sys.systables t WHERE s.schemaid = t.schemaid   and t.tabletype = 'T'   ;

Tables status looked good so this point us to derby tranlog corruption.
To have a new log with correct file structure we have created a new empty derby db with following command

ij> connect 'jdbc:derby:MyDbTest;create=true';

after that we have copied the  log/  from new db to log/ of  wpsdb (all steps with releated backup) and after have copied back the new wpsdb folder the Portal was able to start again !

I like to thank  Andrea Grassi for his blogpost about derby corrupted database recovery guide.

WebSphere How To: fix errore ADMG0011E while Application Server deletion

Working on a WebSphere 8.5.0.2 environment I was unable to delete an Application server getting the error ADMG0011E:

An error occurred while deleting the server. ADMG0011E: An unexpected exception occurred com.ibm.websphere.management.exception. AdminException: Exception in removeTargetFromCUs()..

Inside DMGR and NodeAgent logs I wasn't able to find anything related but inside DMGR FFDC, folder I got a log generated during the deletion time frame with following exceptions:

Caused by: com.ibm.wsspi.management.bla.op.OpExecutionException: CWWMH0121E: Could not read configuration data for composition unit "WebSphere:cuname=<Application_Name>,cuedition=BASE". Error data: com.ibm.wsspi.management.bla.op.OpExecutionException: Enterprise application data for composition unit "WebSphere:cuname=<Application_Name>,cuedition=BASE" could not be read. at com.ibm.ws.management.bla.model.CompositionUnitConfig.read(CompositionUnitConfig.java:293)

After some research  I've found this question on  DeveloperWorks and than I've fixed the issue following this steps:

1. Deployement Manger stop
2. check inside WAS_INSTALL_ROOT/profiles/Dmgr_ProfileName/config/cells/Cell_Name/applications if an  Application_Name.ear folder with same FFDC name Application_Name was present
3. If the folder on step 2 was missing check inside WAS_INSTALL_ROOT/profiles/Dmgr_ProfileName/config/cells/Cell_Name/blas and than delete  Application_Name if present
4. check inside WAS_INSTALL_ROOT/profiles/Dmgr_ProfileName/config/cells/Cell_Name/cus if the  Application_Name folder is present than delete
5. Deployement Manger start
6. Full Nodes Sync from WebSphere console
7. Tried again to delete the  Application server and it worked!

Thanks to  Jeanette Daly for the detailed response provided at this page .. 

IBM WebSphere 9 with Java 8.0 Service Refresh 5 : how to fix sync issue when LTPA token become invalid

During last weeks I was working on  WebSphere Portal 9 environment who runs over a WebSphere Application server 9.0 FP4 with last JVM available 8.0 SR5 .  Every day after some hours I've noted the nodes goes in unsynched status without a clear reason and after a manual SyncNode the syncronization will be restablished until the next day.

After some investigation I noted sync issue was started at the end of the LTPA token with the following errors:

Node Agent:
[11/3/17 17:05:35:849 CET] 0000006b AdminServiceI 3   invoke method throws exception
                                 com.ibm.websphere.management.exception.AdminException: javax.management.JMRuntimeException: ADMN0022E: Access is denied for the isNodeSynchronized operation on NodeSync MBean because of insufficient or empty credentials.

DMGR
[11/3/17 17:05:52:651 CET] 00000105 SecurityHelpe >  isCertPathValidatorException Entry
                                 com.ibm.websphere.security.auth.WSLoginFailedException: Validation of LTPA token failed due to invalid keys or token type.


I've open a PMR and after I've send the log   ( remember to follow the Must gather steps when you set logs to WAS support) IBM support has recognized a similar behavior on  known APAR

IJ00732: JAVA JIT: INCORRECT RESULTS FOR PATTERNS OF SHIFTED AND OR'D BYTES
http://www.ibm.com/support/docview.wss?uid=swg1IJ00732
Due to this JIT failure, sudden LTPA toke validation issue happens and it results into the  synchronization issue.

Would it be possible to see if the issue can be avoided by the JVM entiry?
System administration > Deployment manager > Process definition  > Environment Entries
Name: TR_DisableIORByteSwap
Value: 1

I've followed the suggested workaround and the issue was fixed !
A new JVM 8.0.5.5 now scheduled for Nov 13th  will fix this issue.

Java 6 SE EOL April 2018 - What do you have to know about WebSphere Application Server

Yesterday I've read this article from Dave Hay (thanks) that was remind me the EOS of JAVA 6SE.
The date is set to April 2018 for traditional WebSphere and September 2017 for WebSphere Liberty Profile.

Others WAS version prior 8.5.5 will not have support of new JRE so must be dismissed to maintain System Security. Java 7 will not be the best migration path because have already set EOL to 2019.

This is a table about Java supported version and WebSphere version

For traditional WebSphere 8.5.5 the step required will be upgrade at least to FP9 ,than install Java SE 8 , than set the default SDK to 8.

Obviously the applications must be able to run with the updated SDK . IBM has build a migration toolkit to scan and analyze your Java apps and check compatibility differences.

Regarding IBM collaboration software like IBM Connections , right now only Java 6 SE is supported so probably during next month IBM will release a CR able to support Java 8 except for Connections 4.5 that have EOS April 2018.

To read more details about J6SE EOL please read this article.