Here we go again! After having written and blogged about this topic a fair bit in the past, having too many managers is a problem I still continue to see all too often. A good example is a site I worked with recently; it had 20+ managers just to run the workflow background processes.

Scheduled requests are one of the mainstays of OEBSconcurrent processing.

Also remember some OEBS application monitors provide alertingto remind you when your indexes may need maintenance.

The paper and the PLSQL function can beviewed or downloaded at the following addresses

PDFdocument, 500 Kb
http://www.piper-rx.com/pages/papers/email_validation_part_two.pdf

PLSQL,8 Kb
http://www.piper-rx.com/pages/papers/piper_rx_email_validation.sql

-Gary

This paper is part one (1) of a two (2)part series. Part 1 covers null e-mail addresses in your workflow roles objectsand in part 2 we covers e-mail address validation including a free 12 point PLSQL functionfor validating e-mail addresses.






The paper can be viewed or downloaded atthe following address:
Part 1: Missing e-mail addresses
PDFdocument, 300 Kb

In part 2 of the “Improving the businessefficiency of OEBS workflow” series we will cover email address validation.Part 2 also includes an 12 point email address validation function.

First and foremost this is done via the normal apps Security > User > Define screen.

Whilst the account name (user_name) has to be unique which is enforced by the fnd_user_u2 index, the internal application key is based on the account ID (user_id) not the name

So why would you change an account name?

• A name change by deed poll or marriage
• A new employee has the same name as an ex-employee
• The user name was entered incorrectly
• You want to implement a user naming standard

Scenario: Once a user is no longer with the company, end date the account and change the account name adding an Z_ to the beginning of the user name e.g. The account GPIPER becomes Z_GPIPER. Now the account name GPIPER can be used by a new employee. This has the added benefit of allowing you to filter reports excluding ex-employees, or getting them to sort last in your reports.

Of course you will need to get this approach signed off by Oracle support and internal audit first. Usually, as long as there is consistent documented evidence of the change audit are generally ok

You are not doing anything dodgy here, this is all done via the standard Oracle application screens (Security > User > Define) just select the user, change the users name and commit.

Note: If you change the name of an active account, you will need to reset that user's password and let them know the password you have set and that they will be required to change their password on first connecting with the new name.

Of course you wouldn’t even attempt to do this without checking with Oracle support and you’re your audit team before changing application user names and then testing, testing etc…..

Index maintenance is all too often overlooked and can, over time, have a serious impact on performance. Ever growing indexes account for large amounts of wasted space and failing to perform regular index rebuilds on dynamic application tables is just throwing away performance.

When APPLSYS has over a thousand indexes the OEBS application has over 60,000 indexes (version and module dependant) it often gets too hard to identify what indexes require rebuilding and they simply don’t get rebuilt. Believe it or not, most sites I visit have not rebuilt indexes on even the basic regularly purged objects such as concurrent requests and sign-on audit tables in years.

Example:

A site I recently reviewed generates approximately 35,900 fnd_concurrent _requests records per day which are being purged nightly holding 7 days history on-line. The indexes had not been rebuilt for over 4 years. Thus in this instance the base table should be holding approximately 251,000 records whereas the primary key index fnd_concurrent_requests_u1 would be holding approximately 52 million rows. 99.9% deleted rows. This is an interesting one when you calculate the space the index is holding, I estimate over 1.4 Gb for fnd_concurrent_requests_u1 alone.

In this paper we will provide a simple list of indexes to start with, and a method of identifying if any of these indexes should be rebuilt.

We also cover why you should exercise caution when using an auto index rebuild script you find on the web, some do not readily translate to an OEBS environment

The full paper can be found at http://www.piper-rx.com/pages/tips.html - Look for "Rebuilding indexes in an OEBS environment" in the tips section

Much like the Accounts Receivable ages debtor accounts function in any business, we can age application user accounts that have never connected to the application or have not connected in some time via the full and self-service screens.

Why would we want to do this? Having people who have long left the business retaining active accounts is a security issue, and all the best managed sites have processes in place to check that such accounts are being closed on a timely basis.

It is possible to generate a list of application user accounts that have either never connected or have not connected in the last 120 days. Note the use of the accounting style time buckets current 30, 60, 90, 120 days...this makes the report business user friendly as generally the target use of this type of report will be business users such as your HR department.

The first step is to identify the number of accounts you have that are candidates for review:

SELECT count(*)
  FROM applsys.fnd_user fu
WHERE (fu.last_logon_date is null
               or (sysdate - fu.last_logon_date) >= 120 )
      and (fu.end_date is null
              or fu.end_date > sysdate )
     and fu.user_name not in ('SYSADMIN','GUEST');

The second step is to produce a list of application accounts that are candidates for review:

SELECT fu.user_id,
             fu.user_name,
             substr(fu.description, 1, 40)
             decode(sign(length(fu.description) - 40), 1, '...') description,
             nvl(to_char(fu.last_logon_date, 'DD-Mon-YY HH24:MI'), 'Never Connected')
                       last_connect_date,
             decode(fu.last_logon_date, null, null,
             round( (sysdate - fu.last_logon_date),0) ) days_since_last_connect
  FROM applsys.fnd_user fu
WHERE (fu.last_logon_date is null
               or (sysdate - fu.last_logon_date) >= 120 )
       and (fu.end_date is null
              or fu.end_date > sysdate )
       and fu.user_name not in ('SYSADMIN','GUEST')
ORDER by decode(fu.last_logon_date, null, (sysdate - 10000), fu.last_logon_date),
                fu.user_name;

The output will look something like the following example:


Remember: Not all Aged User Accounts are candidates for end dating:

There are of course exceptions, those accounts that must exist but are never used and those that are used very infrequently. You can exclude these by simply adding these accounts to your SQL statement:

             and fu.user_name not in (‘SYSADMIN’,’GUEST’)

You should run the second report say once per month as part of your normal application maintenance routine and send it to your HR department or whoever manages your application user accounts. It shows the business you are on top of managing the application and the business loves anything to do with security...

All too often I see sites with far too many concurrent manager processes and the site wonders why they have intermittent performance issues. Remember, having more concurrent manager processes does not mean more throughput.

So, how many concurrent manager processes should you have?

Add up the number of standard and custom concurrent manager processes you have. If that value exceeds 2 times the number of CPUs (multi core adjusted) on the box you most likely have too many manager processes. If you are experiencing intermittent performance issues, particularly around high processing times like financial month ends, too many manager processes would most likely be one of your reasons.

The following SQL will list your concurrent managers:

SELECT concurrent_queue_name,
                max_processes,
                running_processes,
                nvl(sleep_seconds,0) sleep_seconds,
               cache_size,
               decode(enabled_flag,
               'Y', 'Enabled', 'N', 'Disabled', 'Unknown' ) status
   FROM applsys.fnd_concurrent_queues
ORDER by decode(enabled_flag, 'Y', 1, 'N', 0, enabled_flag ) DESC,
                    max_processes DESC,
                    decode(concurrent_queue_name,
                                  'FNDICM', 'AA',
                                  'FNDCRM', 'AB',
                                  'STANDARD', 'AC',
                                  concurrent_queue_name);
 
A real world example of too many managers:

A site I reviewed had 54 standard manager processes on a 4 CPU box - 54 / 4 = 13.5

In this example there can be up to 54 concurrent requests running through the standard managers at any time which will just plain flood the CPUs. Generally running two (2) concurrent requests per CPU is sufficient to leave enough overhead for normal processing activities; any more than that and the risk increases of CPU flooding. And you would hope this site does run too many FSG’s... as we all know the damage they can do...

Believe it or not I have even seen 108 standard managers on an 8 CPU box..... Hmmm.....

Flooding the CPUs with concurrent requests leaves very little available CPU for normal user requests. As a result users tend to experience poor performance with their forms etc... It’s basic queuing theory... To make matters worse, these intermittent performance issues tend to occur around peak processing times when the concurrent manager load is at its peak; exactly the time users are clearly very busy and don’t want to be experiencing performance issues.

If you have a “problem” with an excessive number of standard and custom concurrent managers processes, what you need to do is lower the number of processes. This is easier said than done, as once they exist the business is very reluctant to let them go. But it is worth persisting as it will make a difference!

For more information refer to the paper I wrote:

http://www.piper-rx.com/pages/papers/cm101.pdf

Believe it or not, and much to my surprise, this is the most down loaded paper on my web site. Even though I wrote this paper in 2004, it still holds true today.... not much changes in OEBS, and for good reason; stability in accounting and business systems is what businesses want.

The site in presented with over 30 million rows in the fnd_logins table, growing at a rate of approximately 25,000 records per day.

Based on the site’s application activity, the estimated number of rows that should be held in the fnd_logins table should be around 800,000 records (i.e. holding 32 days history on-line). It is expected that this number should reduce post concurrent manager activity review.

The site was running the concurrent program Purge Signon Audit data (FNDSCPRG) daily as part of their normal maintenance program to purge the sign-on audit data and was unaware of the high growth rate in the sign-on audit tables.

On review of the site’s scheduled requests it was found that the Purge Signon Audit data was being run daily with the date argument set to 10-Oct-06, however, “the increment date parameter each run” check box had not been checked when the scheduled request was created. As such the data parameter has not been incrementing with each run, thus with each run the program has been purging all sign-on audit records prior to 10-Oct-06. As a result any record added after that date has not been being purged.

So, since 10-Oct-06 (Over 1,200 days at the time of presentation) the program has been running but purging nothing.

Whilst there are several inherent performance issues, the biggest impact to the application would be when the concurrent program runs. The concurrent program Purge Signon Audit data (FNDSCPRG) uses an un-indexed column start_time in its execution. As such the purge program will execute a full scan on each of the target tables in order to determine the rows to delete. So for this site, that would include a daily full scan of a 30 million to not find any rows to delete.

The case review shows how I would “Fix” the issue including deleting the unwanted rows and setting up a new Purge Signon Audit data scheduled program run.

The full case review can be found at -
http://www.piper-rx.com/pages/case_reviews/case_sign_on_audit.pdf