Design for `mvserver`, a modular, generic application-server framework.

A picture is worth (roughly) one thousand words

Important things not denoted by this picture

Worker threads and "entities" (large functionally independent "chunks" of computation) are homomorphic.
Entities live in shared libraries and are dynamically loaded when registered and used when called.
There's a pool of "listener" threads putting work requests on the queue for the workers to take off. They take RAD (the protocol used for transfer) requests from the network, parse them into work structures, and place them on the work queue. Listener threads are necessary to parallelize the handling of new requests and previous requests.
Worker threads wait on the semaphore before they wait on the mutex. This prevents deadlock.

What does `mvserver` do?

Pool threads: exchange space for time...
Provide a generic entity interface with an abstract concept of work.
Has a main() that (presumably) all application-servers need.
Provide modularity for entities, allows them to be plugged in or left out.

What pieces of data will any server implemented using the mvserver framework need?

hostname
upper/lower limits on worker and listener thread pool size
read port
argument parsing routines
entity registration
registration of the arguments the entities will expect

English description

Thread pooling

Some application servers launch one thread for each incoming connection. This approach causes each new request to cost at least the cost for spawning, initializing and destroying a new thread, a considerable cost. Also, this practice could allow a denial-of-service attack to succeed.

By pooling threads, we save the cost of thread creation and destruction at each request point. Also, it would be possible to analyze and reject work in the queue if the queue becomes too long (shed load). No such analysis is possible (or such analysis is more difficult) when automatically spawning a thread for each incoming connection. This time is purchased at the cost of having more memory allocated in the form of worker thread stacks. We must also pay the cost of synchronization: exclusive access to the shared queue and maintaining the semaphore to notify worker threads when there is more work.

Abstract interface

In the new model, each entity is a shared library with an advertised entry point. There are two types of entities, RAD and XML entities. RAD entities are lighter-weight than XML entities but do not offer typed arguments (all arguments are considered strings).

Each RAD entity will have the same signature,

void f(work_t *work)

(see 'The abstract notion of "work"' for work_t) which will make entities easier to use and write and allow factoring of common code. This entry point will be associated with an entity name (what we think of as an entity) by calling the procedure named RegisterRADEntity with the entity name. The name of the entry point is inferred. RegisterRADEntity will determine the proper function address through the use of dladdr.

Each XML entity will have the same signature,

void f(XMLRPC_REQUEST pRequest, work_t *work)

(see 'The abstract notion of "work"' for work_t, and xmlrpc docs for a description of XMLRPC_REQUEST) which will make entities easier to use and write and allow factoring of common code. This entry point will be associated with an entity name (what we think of as an entity) by calling the procedure named RegisterXMLEntity with the entity name. The name of the entry point is inferred. RegisterXMLEntity will determine the proper function address through the use of xmlrpc.

The abstract notion of "work"

Work, in the mvserver, is an ADT, work_t. In order for the mvserver to do work, it gives one of its worker threads a pointer to one of these structures, and has it invoke the entity registered (through RegisterAllEntities) for the entity named in the RAD request with a pointer to a work_t structure as an argument.

Treat work_t as an opaque type, using only the functions in work.h:

/* taken from work.h,v 1.6 2000/07/11 01:15:29 mooreb Exp */
work_t *work_new_work(void);
int work_get_unique_id(work_t *work);
const char *work_get_name(work_t *work);
HashTable work_get_entity_arguments(work_t *work);
int work_get_num_entity_arguments(work_t *work);
const char *work_get_rad_body(work_t *work);
int work_get_rad_body_length(work_t *work);
int work_get_socket(work_t *work);
int work_get_is_xml(work_t *work);
int work_get_wants_response(work_t *work);
void work_set_unique_id(work_t *work, int unique_id);
void work_set_name(work_t *work, const char *external_entity_name);
void work_set_entity_arguments(work_t *work, HashTable ht);
void work_set_num_entity_arguments(work_t *work, int num_entity_arguments);
void work_set_rad_body(work_t *work, const char *rad_body);
void work_set_rad_body_length(work_t *work, int rad_body_length);
void work_set_socket(work_t *work, int socket);
void work_set_is_xml(work_t *work, int is_xml);
void work_set_wants_response(work_t *work, int wants_response);
void work_print_work(work_t *work);
void work_free_work(work_t *work);
void work_set_output_string_func(work_t *work, output_string_func f);
void work_set_output_header_func(work_t *work, output_header_func f);
void work_send_output(work_t *work, const char *s);
void work_send_header(work_t *work, const char *name, const char *value);

Answers to your burning questions

How do I take this and add my own code to make a useful server?
An example is in server_framework/example. Recursively copy this directory to a new location and modify for your own needs. You will at least need to change the include directives in the C files to point to the location of the framework, relative to the new location of the server.
The mvserver framework itself is in src/server_framework/src. You should not need to modify anything in this directory.
In detail: You take the provided framework (as is) and add your own code to make a useful server by implementing several functions:
- RegisterAllEntities, (with calls to RegisterRADEntity/RegisterXMLEntity),
- RegisterEntityArguments, (with calls to RegisterEntityArgument),
- RegisterCommandLineArguments, (with calls to RegisterEntityArgument),
- ProcessCommandLineArguments, (with calls to ProcessCommandLineArgument), and
- DoInitialization.
That should be it. In particular, you shouldn't need to modify anything in mvserver.c or mvserver.h. I may have made a mistake in implementing these things, but they should be generic, extendable, and meet all your framework needs. If it doesn't, tell me, and I'll fix it.
But without changing main() How do I set new values for the hostname or the read port or the number of threads in the pool?
There are a number of functions implemented inside the framework that a server implementing the framework is allowed to call. These functions include:
- void SetHostName(const char *hostname);
- void SetReadPort(const char *readport);
- void SetMinimumWorkerThreadPoolSize(const char *threadpool_size);
- void ParsePrefFile(const char *fname);
What are the command line arguments, and how do I add my own extensions?
The command line arguments are whatever are registered by RegisterCommandLineArguments, and you add your own extension by implementing RegisterCommandLineArguments, and ProcessCommandLineArguments,
What's the deal with the preferences file?
The default preferences file is called by the name defined by the macro DEFAULT_PREF_FILE defined in prefs.h. It is parsed before the command line arguments are parsed. If the command line names another preferences file, it is parsed. Any arguments in this file override any preferences in the default preferences file or previous command line arguments. Any command line arguments following either preferences file override values defined in the preferences files or earlier in the command line.
What kind of throughput can I expect (burst and sustained)?
I can currently with four worker threads in the threadpool sustain roughly more than 6000 requests per second on a Sun 4500. This was measured using the new framework and the "client" program in the rad directory. I don't know how to measure burst rate, or I'd give you a figure for that, too.
Why can't I just add my line of code into your main.c?
You can't do that because that would violate the modularity of this server framework. Remember, lots of people are using this framework for lots of things. If you find yourself wanting to do things in main, you might be able to do them in the _init function that gets called when your shared library is loaded. Or maybe it could go in DoInitialization. Or it could be that I've left something critical out. Let me know and we can consider extending the framework.

Brian Moore

Last modified: Wed Mar 21 17:20:23 PST 2001

Design for mvserver, a modular, generic application-server framework.