I am getting Error 11 (EAGAIN No more processes)
A fork failed because the process table of the system is full or the user is not allowed to create any more processes in audit_spec file.

How to rectify this error?
Do I need to reboot the Server or is there any other option without rebooting the server?

Rebooting or not depends on how flexible (read: dynamic) your kernel is. If you are working on AIX then you may be able to tweak it online as AIX's kernel is pretty dynamic.

Parameters (or variants) you should check are: MAXUPRC, MAXUSERS, NPROC (these are the HP-UX conventions, don't know the AIX ones)

Can u please guide me as how to set these values for the parameters you have mentioned?
I am attaching a Unix System File for your reference.
Thanks a lot for your guidance.

I am not familiar enough with AIX to give you good guidance on which parameters to tweak. However what you should do is read through the installation guides for Baan and your database engine where there are recommendations for kernel parameters. With these recommendations and your current user requirements you should be able to come up with an acceptable kernel configuration. From the system file you attached I can only find custom settings regarding semaphores and shared memory. That might indicate that you are using defaults for all the rest (process parameters etc) and usually they are too low.