Bandwidth

Test the bandwidth between two nodes. More...

Functions
void	amok_bw_init (void)
	module initialization; all participating nodes must run this
void	amok_bw_exit (void)
	module finalization
void	amok_bw_test (gras_socket_t peer, unsigned long int buf_size, unsigned long int msg_size, unsigned long int msg_amount, double min_duration, double *sec, double *bw)
	bandwidth measurement between localhost and peer
void	amok_bw_request (const char *from_name, unsigned int from_port, const char *to_name, unsigned int to_port, unsigned long int buf_size, unsigned long int msg_size, unsigned long int msg_amount, double min_duration, double *sec, double *bw)
	request a bandwidth measurement between two remote peers
double *	amok_bw_matrix (xbt_dynar_t hosts, int buf_size_bw, int msg_size_bw, int msg_amount_bw, double min_duration)
	builds a matrix of results of bandwidth measurement
void	amok_bw_saturate_start (const char *from_name, unsigned int from_port, const char *to_name, unsigned int to_port, unsigned int msg_size, double duration)
	Ask 'from_name:from_port' to stop saturating going to to_name:to_name.
void	amok_bw_saturate_begin (const char *to_name, unsigned int to_port, unsigned int msg_size, double duration, double *elapsed, double *bw)
	Start saturating between the current process and the designated peer.
void	amok_bw_saturate_stop (const char *from_name, unsigned int from_port, double *time, double *bw)
	Ask 'from_name:from_port' to stop any saturation experiments.

---

Detailed Description

Test the bandwidth between two nodes.

This module allows you to retrieve the bandwidth between to arbitrary hosts and saturating the links leading to them, provided that they run some GRAS process which initialized this module.

Bandwidth measurement

Retrieving the bandwidth is usually done by active measurment: one send a packet of known size, time how long it needs to go back and forth, and you get the bandwidth in Kb/s available on the wire.

This is not as easy as it first seems to do so in GRAS. The first issue is that GRAS messages can get buffered, or the receiver cannot be waiting for the message when it arrives. This results in extra delays impacting the measurement quality. You thus have to setup a rendez-vous protocol. The second issue is that GRAS message do have an header, so figuring out their size is not trivial. Moreover, they get converted when the sender and receiver processor architecture are different, inducing extra delays. For this, GRAS provide the so-called measurement sockets. On them, you can send raw data which is not converted (see Using measurement sockets).

Solving all these problems is quite error prone and anoying, so we implemented this in the current module so that you don't have to do it yourself. The API is very simple. Use amok_bw_test() to get the BW between the local host and the specified peer, or amok_bw_request() to get the BW between two remote hosts. The elapsed time, as long as the achieved bandwidth is returned in the last arguments of the functions.

All sizes are in bytes. The buf_size is the size of the buffer (this is a socket parameter set automatically). The exp_size is the amount of data to send during an experiment. msg_size is the size of each message sent. These values allow you to study phenomenon such as TCP slow start (which are not correctly modelized by SURF, yet). They are mimicked from the NWS API, and default values could be buf_size=32k, msg_size=16k and exp_size=64k. That means that the socket will be prepared to accept 32k in its buffer and then four messages of 16k will be sent (so that the total amount of data equals 64k). Of course, you can use other values if you want to.

Fig 1: AMOK bandwidth measurement protocol.

Link saturation

You sometimes want to try saturating some link during the network related experiments (at least, we did ;). This also can turn quite untrivial to do, unless you use this great module. You can either ask for the saturation between the current host and a distant one with amok_bw_saturate_begin() or between two distant hosts with amok_bw_saturate_start(). In any case, remember that gras actors (processes) are not interruptible. It means that an actor you instructed to participate to a link saturation experiment will not do anything else until it is to its end (either because the asked duration was done or because someone used amok_bw_saturate_stop() on the emitter end of the experiment).

The following figure depicts the used protocol. Note that any handshaking messages internal messages are omitted for sake of simplicity. In this example, the experiment ends before the planned experiment duration is over because one host use the amok_bw_saturate_stop() function, but things are not really different if the experiment stops alone. Also, it is not mandatory that the host calling amok_bw_saturate_stop() is the same than the one which called amok_bw_saturate_start(), despite what is depicted here.

Fig 2: AMOK link saturation protocol.

---

Function Documentation

void amok_bw_test	(	gras_socket_t	peer,
		unsigned long int	buf_size,
		unsigned long int	msg_size,
		unsigned long int	msg_amount,
		double	min_duration,
		double *	sec,
		double *	bw
	)

bandwidth measurement between localhost and peer

• peer: A (regular) socket at which the the host with which we should conduct the experiment can be contacted
• buf_size: Size of the socket buffer. If 0, a sain default is used (32k, but may change)
• msg_size: Size of each message sent.
• msg_amount: Amount of such messages to exchange
• min_duration: The minimum wanted duration. When the test message is too little, you tend to measure the latency. This argument allows you to force the test to take at least, say one second.
• sec: where the result (in seconds) should be stored. If the experiment was done several times because the first one was too short, this is the timing of the last run only.
• bw: observed Bandwidth (in byte/s)

Conduct a bandwidth test from the local process to the given peer. This call is blocking until the end of the experiment.

If the asked experiment lasts less than min_duration, another one will be launched (and others, if needed). msg_size will be multiplicated by MIN(20, (min_duration / measured_duration) *1.1) (plus 10% to be sure to eventually reach the min_duration). In that case, the reported bandwidth and duration are the ones of the last run. msg_size cannot go over 64Mb because we need to malloc a block of this size in RL to conduct the experiment, and we still don't want to visit the swap. In such case, the number of messages is increased instead of their size.

Results are reported in last args, and sizes are in byte.

Warning:

: in SimGrid version 3.1 and previous, the experiment size were specified as the total amount of data to send and the msg_size. This was changed for the fool wanting to send more than MAXINT bytes in a fat pipe.

void amok_bw_request	(	const char *	from_name,
		unsigned int	from_port,
		const char *	to_name,
		unsigned int	to_port,
		unsigned long int	buf_size,
		unsigned long int	msg_size,
		unsigned long int	msg_amount,
		double	min_duration,
		double *	sec,
		double *	bw
	)

request a bandwidth measurement between two remote peers

• from_name: Name of the first peer
• from_port: port on which the first process is listening for messages
• to_name: Name of the second peer
• to_port: port on which the second process is listening (for messages, do not give a measurement socket here. The needed measurement sockets will be created automatically and negociated between the peers)
• buf_size: Size of the socket buffer. If 0, a sain default is used (32k, but may change)
• msg_size: Size of each message sent.
• msg_amount: Amount of such data to exchange
• sec: where the result (in seconds) should be stored.
• bw: observed Bandwidth (in byte/s)

Conduct a bandwidth test from the process from_peer:from_port to to_peer:to_port. This call is blocking until the end of the experiment.

Warning:

: in SimGrid version 3.1 and previous, the experiment size were specified as the total amount of data to send and the msg_size. This was changed for the fool wanting to send more than MAXINT bytes in a fat pipe.

Results are reported in last args, and sizes are in bytes.

double* amok_bw_matrix	(	xbt_dynar_t	peers,
		int	buf_size_bw,
		int	msg_size_bw,
		int	msg_amount_bw,
		double	min_duration
	)

builds a matrix of results of bandwidth measurement

Warning:

: in SimGrid version 3.1 and previous, the experiment size were specified as the total amount of data to send and the msg_size. This was changed for the fool wanting to send more than MAXINT bytes in a fat pipe.

void amok_bw_saturate_start	(	const char *	from_name,
		unsigned int	from_port,
		const char *	to_name,
		unsigned int	to_port,
		unsigned int	msg_size,
		double	duration
	)

Ask 'from_name:from_port' to stop saturating going to to_name:to_name.

Parameters:

from_name,:	Name of the peer we are asking to do a experiment with (to_name:to_port)
from_port,:	port on which the process we are asking for an experiment is listening (for message, do not give a raw socket here. The needed raw socket will be negociated between the peers)
to_name,:	Name of the peer with which we should conduct the experiment
to_port,:	port on which the peer process is listening for message
msg_size,:	Size of each message sent.
duration,:	How long in maximum should be the saturation.

Ask the process 'from_name:from_port' to start to saturate the link between itself and to_name:to_name.

void amok_bw_saturate_begin	(	const char *	to_name,
		unsigned int	to_port,
		unsigned int	msg_size,
		double	duration,
		double *	elapsed_res,
		double *	bw_res
	)

Start saturating between the current process and the designated peer.

Note that the only way to break this function before the end of the timeout is to have a remote peer calling amok_bw_saturate_stop to this process.

If duration=0, the experiment will never timeout (you then have to manually stop it).

If msg_size=0, the size will be automatically computed to make sure that each of the messages occupy the connexion one second

void amok_bw_saturate_stop	(	const char *	from_name,
		unsigned int	from_port,
		double *	time,
		double *	bw
	)

Ask 'from_name:from_port' to stop any saturation experiments.

Parameters:

from_name,:	Name of the peer we are asking to do a experiment with (to_name:to_port)
from_port,:	port on which the process we are asking for an experiment is listening
time,:	the duration of the experiment
bw,:	the achieved bandwidth