EM12c : Login to GUI with the correct password causes authentication failure

So the other day I was trying to log in to my EM12c R4 environment with the SSA_ADMINISTRATOR user, and I got the error:

“Authentication failed. If problem persists, contact your system administrator”

I was quite sure that the password that I had was correct, so I tried with the SYSMAN user and had the same error. I still wanted to verify that I had the correct password , so I tried with the SYSMAN user to log in to the repository database, and was successful, so I know something was wrong there.

SQL> connect sysman/
Enter password:
Connected.

So I went to the<gc_inst>/em/EMGC_OMS1/sysman/log/emoms.log and saw the following error

2015-05-18 21:22:06,103 [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR audit.AuditManager auditLog.368 - Could not Log audit data, Error:java.sql.SQLException: ORA-14400: inserted partition key does not map to any partition
ORA-06512: at "SYSMAN.MGMT_AUDIT", line 492
ORA-06512: at "SYSMAN.MGMT_AUDIT", line 406
ORA-06512: at line 1

Which led me to believe that the JOB_QUEUE_PROCESSES was set to 0, but it wasn’t the case, since it was set to 50. Though, this is actually an incorrect limit, so I bumped it up to 1000 and tried to rerun the EM12c repository DBMS Scheduler jobs as per the documentation in 1498456.1:

SQL> show parameter JOB_QUEUE_PROCESSES
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
job_queue_processes integer 50
SQL> alter system set JOB_QUEUE_PROCESSES=1000 scope = both;
System altered.
SQL> show parameter both
SQL> show parameter job
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
job_queue_processes integer 1000
SQL> connect / as sysdba
Connected.
SQL> alter system set job_queue_processes = 0;
System altered.
SQL> connect sysman/alyarog1605
Connected.
SQL> exec emd_maintenance.remove_em_dbms_jobs;
PL/SQL procedure successfully completed.
SQL> exec gc_interval_partition_mgr.partition_maintenance;
PL/SQL procedure successfully completed.
SQL> @$OMS_HOME/sysman/admin/emdrep/sql/core/latest/admin/admin_recompile_invalid.sql SYSMAN
old 11: AND owner = upper('&RECOMPILE_REPOS_USER')
new 11: AND owner = upper('SYSMAN')
old 26: dbms_utility.compile_schema(upper('&RECOMPILE_REPOS_USER'),FALSE);
new 26: dbms_utility.compile_schema(upper('SYSMAN'),FALSE);
old 41: WHERE owner = upper('&RECOMPILE_REPOS_USER')
new 41: WHERE owner = upper('SYSMAN')
old 84: AND owner = upper('&RECOMPILE_REPOS_USER')
new 84: AND owner = upper('SYSMAN')
old 104: AND ds.table_owner = upper('&RECOMPILE_REPOS_USER')
new 104: AND ds.table_owner = upper('SYSMAN')
PL/SQL procedure successfully completed.
PL/SQL procedure successfully completed.
SQL> connect / as sysdba
Connected.
SQL> alter system set job_queue_processes = 1000;
System altered.
SQL> connect sysman/
Enter password:
Connected.
SQL> exec emd_maintenance.submit_em_dbms_jobs;
PL/SQL procedure successfully completed.
SQL> commit;
Commit complete.

After this I bounced the OMS, but still kept getting the same error. And though it fixed the scheduler jobs, I was now seeing the following error in the emoms.log:

2015-05-18 22:29:09,573 [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] WARN auth.EMRepLoginFilter doFilter.450 - InvalidEMUserException caught in EMRepLoginFilter: Failed to login using repository authentication for user: SSA_ADMIN
oracle.sysman.emSDK.sec.auth.InvalidEMUserException: Failed to login using repository authentication for user: SSA_ADMIN

So what I did was an update to the SYSMAN.MGMT_AUDIT_MASTER table and ran the procedure MGMT_AUDIT_ADMIN.ADD_AUDIT_PARTITION as was stated in document id 1493151.1:

oracle $ sqlplus
 
Enter user-name: sysman
Enter password:
SQL> update mgmt_audit_master set prepopulate_days=5 where prepopulate_days is null;
1 rows updated.
SQL> select count(1) from mgmt_audit_master where prepopulate_days is null;
COUNT(1)
----------
0
SQL> exec mgmt_audit_admin.add_audit_partition;
PL/SQL procedure successfully completed.
SQL> commit;
Commit complete.

Once I did this, I was able to login with all my EM12c administrators without any issues:

[email protected] [emrep] /home/oracle
oracle $ emcli login -username=ssa_admin
Enter password
Login successful

Conclusion

Even though the JOB_QUEUE_PROCESSES were not set to 0, it was the cause that it was failing, as it was a low value for this parameter. Thus, be careful when setting up this parameter, be sure to follow the latest installation guidelines.

Note– This was originally published on rene-ace.com