Xen 
 
Home About Xen.org Xen Xen Summit Wiki Mailing List Bug Tracker Xen Downloads
 
   
 

xen-devel

Re: [Xen-devel] xenconsoled CPU denial of service problem

To: Keir Fraser <Keir.Fraser@xxxxxxxxxxxx>
Subject: Re: [Xen-devel] xenconsoled CPU denial of service problem
From: "Daniel P. Berrange" <berrange@xxxxxxxxxx>
Date: Tue, 29 Aug 2006 16:59:00 +0100
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Tue, 29 Aug 2006 08:59:37 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <C1191742.EF0%Keir.Fraser@xxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20060828180224.GG862@xxxxxxxxxx> <C1191742.EF0%Keir.Fraser@xxxxxxxxxxxx>
Reply-to: "Daniel P. Berrange" <berrange@xxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Mutt/1.4.1i
On Mon, Aug 28, 2006 at 09:57:22PM +0100, Keir Fraser wrote:
> On 28/8/06 7:02 pm, "Daniel P. Berrange" <berrange@xxxxxxxxxx> wrote:
> 
> > Does anyone know of any alternative approach to detecting whether the fd
> > for the master end of a psuedo-TTY, has a its end slave open / active ?
> > Without being able to detect this I don't see any good way to avoid the DOS
> > attack in the general case - only other option would be to start dropping
> > data once > a certain rate, but this isn't really very desirable because
> > there are (debug) scenarios in which you really do want the ability to
> > capture all data.
> 
> The protocol has flow control. If we rate-limited xenconsoled consumption of
> data from each domU ring, we would limit resource consumption in dom0 and
> not lose any data (since the domU will simply buffer it internally).

Ah, that's very handy. Looking at the xenconsoled code I think I've worked
out how to slow things down - although I'm not entirely sure I'm correctly
activating the throttling - basically I'm delaying the calls to the methods
xc_evtchn_notify & xc_evtchn_unmask after consuming data - is this the
correct approach ?

I'm attaching a patch which implements this scheme. In the buffer_append
method we keep a cumulative count of how much data we've consumed. Every
time if exceeds N bytes, we calculate  the period over which those N bytes
were received. If this period is < than a threshold (ie fast) then I insert
a small delay before notifying & unmasking the event channel.

The patch sets

  - data size 5 kb
  - period 200 ms
  - delay 200 ms

ie, if we receive 5 kb of data in less than 200 ms, we delay for 200 ms
before allowing more. These constants are #define'd for easy tuning, the
current values are fairly conservative / low data rate. Any thoughts on
what an appropriate data rate to allow from DomU is ?

Finaly question - in the handle_ring_read() method is the port returned
by xc_evtchn_pending guarrenteed to be same as port we already have a
reference to in 'struct domain' local_port field ? If so I can remove 
the saved reference 'limited_port' I added in 'struct buffer'

For testing purposes just pick a DomU and run

  while /bin/true ; do echo t > /proc/sysrq-trigger ; done

Ordinarily this would cause Dom-0/xenconsoled to consume 100%. With this
patch applied there is negligable CPU time consumed by Dom-0/xenconsoled.

(If you increase delay to 2,000 ms you can visually see that no data is
 being lost as a result of the delays). 

  Signed-off by: Daniel P. Berrange <berrange@xxxxxxxxxx> 

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

Attachment: xen-console-ratelimit-3.patch
Description: Text document

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel