Profound UI Instance performance issues

[thutcheson]

We have a client that is running our software with Profound UI and is experiencing occasional performance issues for all users. When it happens it happens to all and it completely stops their business from processing. We have checked and don't see any errors in the logs nor any issues with CPU usage. The memory pool seems more than sufficient for handling the QHTTPSVR (hovering at about 168000 M). I read the post on httpd.conf settings with respect to ThreadsPerChild but I still have a question on that as well as the MaxCGIJobs property and memory usage.

The client has about 2000 instance jobs at any one time for their users. Understanding that only a portion of those are active at any one time:
1. Does ThreadsPerChild = 40 seem small? The performance issue happens for all users so it makes me wonder if this is not an issue of latency per user.
2. We have many CGI program calls in our app. What are your recommendations on MaxCGIJobs being increased from 1000 (the default)? What would be the repercussions? I couldn't find anything in the forums or docs.
3. Even though there are jobs that are not active - do those instance jobs waiting on a request take up system memory?

Thanks! Anything that helps us figure this out would be much appreciated!

[Scott Klement]

What makes performance issues difficult is that there are sooooo many potential things that could cause them. There really isn't a simple "how-to" (or "cookie-cutter") approach to fixing performance issues that I know of. You have to really study things, narrow down what might be causing the issue, etc. i.e. find it the "hard way", and then once you know what it is, take appropriate steps.

The client has about 2000 instance jobs at any one time for their users. Understanding that only a portion of those are active at any one time:
1. Does ThreadsPerChild = 40 seem small? The performance issue happens for all users so it makes me wonder if this is not an issue of latency per user.

For 2000 concurrent sessions, yes, I'd say 40 seems small. I would try upping it to about 250 and see if that helps. Making this too high can also cause performance problems, so some amount of trial & error will be needed. So change this (by itself), and monitor the response times. Also, in the IBM Web Admin GUI there's a "Real-time Server Statistics" that you should watch. See what the available thread count is at peak times... then tweak accordingly.

2. We have many CGI program calls in our app. What are your recommendations on MaxCGIJobs being increased from 1000 (the default)? What would be the repercussions? I couldn't find anything in the forums or docs.

This (along with MaxPersistentCgi) are the maximum number of simultaneous jobs that can be run. If you have 2000 jobs, but the max is set to 1000, then half of your users should be getting '503 Service Unavailable' errors since the system can only start 1000 jobs.

Each Profound session (Rich Display or Genie) will require exactly one Persistent CGI job. So if you want to allow 2000 concurrent jobs (or maybe 2500 to be safe) you'll want MaxPersistentCGI to be 2500. Profound UI will also fire off some 'normal' CGI jobs for things like SQL queries, custom urls, designer sessions, etc, So you'll want MaxCgiJobs to be larger than MaxPersistentCGI..

So let's say you have 2000 concurrent users, and you want each user to be able to have two sessions active at the same time. You'd want to set things up like this:

Code: Select all

MaxPersistentCGI 4000
MaxCGIJobs 4400

Keep in mind, also, that each Atrium tab/window that's open counts as a separate session, so if your users routinely have 10 tabs open, these numbers should be much higher (10 x 2000... or whatever.)

3. Even though there are jobs that are not active - do those instance jobs waiting on a request take up system memory?

Not sure what you mean by "jobs that are not active." If a user has a window open that's connected to a Profound UI session, then the job is active, and is using memory, threads, etc.

If the session expires or they close the window, then the corresponding server job should automatically end. (If not automatically -- for example, if the PC powers down without alerting the server that it's closing the sessions, or if the browser crashes, etc -- then it will end when the session times out.) These do not use resources.

If it's a job that previously handled a stateless query (like an SQL query from a database-driven component) but that query has finished running, then the job may stay active but will be reused for the next user. In that case, it will be using some memory, but it will be a minimal amount. It will not count against the number of threads active, since it's available to process new requests. (So is an "available thread")

I will attach a powerpoint from IBM that discusses some performance tips for the HTTP server.

There is also a Redbook that discusses performance (see Chapter 10):
http://www.redbooks.ibm.com/abstracts/sg246716.html

Hope that helps

[thutcheson]

Thanks Scott for the response! That is great information. We have been looking through the tuning options for that last two days. We have some initial changes that we can start with and a number of HTTP tuning options that we can try out over time. We haven't seen the 503 errors but, after your explanation, I wonder why. Will keep an eye out. Thanks again!