locigno disi.unitn.it
Past academic year courses: bottom page
Welcome to the 2014-15 web site of Simulation and Performance Evaluation. This is a dynamic, one-way communication channel, you can interact (with some parsimony, please!) sending me e-mails. I know more modern tools can be adopted, but I'm lazy, and I don't want to adopt a moodle platform or open a facebook prfile for the course.
The billboard is used to convey messages and announcements also to those that cannot be in class, so please check it regularly.
Slides and other teaching material (including assignements) will be posted here in due time.
Tuesday 9:00--11:00 and Thursday 14:00--16:00, Room A203
Please, refer to ESSE3 and the syllabus relative to Simulation and Performance Evaluation for the official course content.
This said, let's try to understand a bit better the course goals and the course content, which, up to some point, will also be twisted and tweked based on how we came to know each other during the first few lessons. As mentioned in the general page of the course site, Performance Evaluation (PE for short) is central to all systems design, and ICT systems are no exception. Event driven simulations of just Simulations to be brief, are one of the means to evaluate performance, but since very often the term "simulation" is used with different meanings, and since it is unfortunately widespread to think about simulations in bery different terms, we explicit it in the title that Simulation techniques and methodology are part of this course.
The main objective of the course is defining what PE means, what are the applications areas, and why it is (or should be) a centerpiece of any system design ... including computers, software, the Internet, and applications in general. Very often performance is just seen as speed ("how fast is your system"?), but it is not a great intellectual feat to understand that speed alone is not sufficient: even a racing car can be the fastest of the lot, but if it breaks down it does not win the race! So reliability is part of performance, availability too, and dependability even more, specially for mission-critical systems.
One of the major issues we will have to clarify is that performance is rarely (if ever) a simple number, but it has to be expressed in stochastic terms, and often as complex vectors of time-dependent variables, and this explain why the starting point of our course is some reasonable knowledge of Probability and Statistics (which in turn imply some reasonable knowledge of Analysis). And we give for granted that, as you all are graduate students of Computer Science, you are all much better than me in programming techniques and software engineering, so that the implementation of simple algorthms, drawting some graphs and manipulating matrices and programs outputs is not a problem.
We will try to make a logical path that starts froms systems, simple or complex as they are, understand, given the system, what are the relevant metrics that define their perfromance, and what are the dependencies between different metrics and different subsystems. Once we have understood what are the metrics of interest and their relationships, we have to take a tough decision to evaluate performance: shall we measure them on the real system, simulate the system, or build an analytical model?
We will start from measures, or better, from the interpretation of measure points and from the design of a correct experimental setup do do the measures. In some cases we might use a simulator as if it were a real system, because this gives us more flexibility ... and makes experiments much, much cheaper!
Next we will introduce simulations, understand what they really are from a mathematical point of view, and learn the basics to design a simulator.
Finally we will make some examples of systems that are prone to an analytical modeling and how these models can be solved to yield results that are otherwise impossible to obtain.
We do not have an official textbook, because the course covers the enormously vast topic of Perfroamcne Evaluation with a "breadth first" approach, so that we would end up using chapters of many different books, and it is not worth buying them. However, I report here some selected books that for one reason or another are relevant for the course ... and try to explain why.
Book No. 1:
K.S. Trivedi,
"Probability and Statistics with Reliability, Queuing, and Computer Science Applications",
Wiley-Interscience; 2001
ISBN: 0-471-33341-7
can be bought on-line AT Amazon or directly from the Wiley Web Site .
2-3 copies are available in the University and Faculties Libraries;
The first five Chapters plus Chapter 10 of this book contain all the background in probability and statistics we need for the course. Indeed, I think there is much more than what you already know, but I do not have control on the courses you took at the Bachelor, thus I cannot be more precise. In conclusion: don't be too scared if some of the material looks new or if some details are beyond your knowledge, we'll rehearse these more advanced parts. If instead you have problems with Bayes rule, feel uncomfortable with percentiles and find difficult to compute the probability that an exponential RV with mean m is larger than 2m, then it is probably better to devote some time to study these Chapters.
Book No. 2:
R. Jain,
"The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling,"
Wiley- Interscience, 1991,
ISBN:0471503361
This book is very old, so I do not advise to buy it, but it is still the only one I'm aware of that tackles the entire range of Perfromance Analysis techniques in the context of Computer Science, thus it is a good reference for the subject as a whole, while taking examples are "real cases" is today a bit outdated.
Book No. 3:
G. Bolch, S. Greiner, H. de Meer, K.S. Trivedi,
"Queueing Networks and Markov Chains : Modeling and Performance Evaluation With Computer Science Applications",
Wiley-Interscience; 1998
ISBN: 0-471-19366-6
(can be bought on-line AT Amazon or directly from the Wiley Web Site)
The following is a very good and advanced book on the use of Markov Chains and Queueing Networks for performance evaluation, but it is definitely not needed for our course, where analytical modeling will be at most 1/3 of the course.
The exam is composed of the assignments we give you during the course through the moodle platform via ESSE3, plus and oral discussion and colloquium. All assignments are individual, meaning that each of you have to deliver a report (often limited in length, to avoid that you write too much). There will be 3 assignments during the course, each of which should not take more than 8-10 hours to complete. Further details will be given later on.
The oral discussion can include exercises, clarifications on the assignements and reports and theoretical, open questions as well. The outcome is a blend of the quality of your reports and the oral discussion, with "more weight" on the assignments if the oral confirms the result; however, you cannot come with assignments perfectly done to the oral, and then be unable to even discuss what is their meaning, in this case the oral dominates, and you may even fail the exam.
All reports must be written in English and also the clarity of the language is part of the evaluation, while the oral can be taken both in English or in Italian as you prefer.
Michele Segata and Luca Baldesi will help me with assignments and their evaluation.
Forthcoming Dates
Refer to ESSE3 for the remaining burocracy.
Please remember to register for the exam before delivering your material and reserve for the oral discussion!
The slides are provided to help taking notes. "Reading" the slides is definitely not enough to study the course. Additinal material, if present, normally refers to somthing we discussed in class, but may have different notation or cover more than we actually did.
The assignments are parte of the exam, indeed a large part of it. We have three different assignemnts, each requiring 10-20 hours of work ... provided you have already studied the relative theory! There are three assignemnts, one on measures, one on simulation, and one on modeling. Each one assignes up to 11 points, so that overall you can "collect" 33 points, that then you have to confirm with the oral discussion.
sudo apt-get install r-base
sudo port install RIf you don't have MacPorts, you can download R binaries from the official R website
The most obvious place to query, ask for clarifications, rise doubts and so on is the class, so that everybody can intervene and be aware of clarifications. Further minor queries can be done at the end (or before) the lesson.
If additional time is needed for complex queries or long problems send me an e-mail for an appointment. Try to specify the reason in the e-mail, this helps me preparing the answer and helps you formalizing it. I do not set up a formal receiving hour: you can come at any time, just take an appointment.
Please, avoid dropping-by the office. Normally I cannot receive you on the fly and the only result is a waste of time. If you have small quick doubts I can also try to solve them via e-mail, so that you don't need to come to my office for a 30 second answer!
locigno disi.unitn.it