[Prev][Next][Index][Thread]

Re: Reliability with redundancy



Jim Fitzpatrick <jim@jfitz.demon.co.uk> wrote:
>nick@vu-vlsi.ee.vill.edu (Nick Pine) wrote:
>
>> A = MTBF/(MTTF+MTTR)
>
>no it isn't, A=MTTF/(MTTF+MTTR), since MTBF=(MTTF+MTTR)

Oops. Yes. Thanks for correcting this, Jim. Gee, somebody actually reads
these postings :-) Of course, the numbers are almost the same. BTW, there
was also a wrong sign in the original, and I've added a bit more explanation:

>>U = 1-A = 0.0008, which is approximately the MTTR/MTTF.

>>P(0) = 1/(M^2/L^2+2M/L+1).
                   ^
                   | this was a - , in error.

>>We could do this calculation exactly, but the first term in the denominator,
>>M^2/L^2, is over a million, and the second term is 2,500 and the third is 1,
>>which is how these calculations usually turn out, so we can approximate
>>P(0) = 1/(M^2/L^2) = (L/M)^2 = (MTTR/MTTF)^2. 

Sometimes I leave errors in postings to see if people read them closely.
Few do. More often I don't see them myself until later.

>>Most people are used to better electrical service than this
>
>how do you know? I have no idea how reliable my home electrical suppy
>is since I spend a large proportion of my year working 25 miles away.
>I have no idea how reliable my work electrical supply is because I
>spend a large proportion of my time at hone asleep! Unless you are in
>one place 24 hours a day 365 days a year with electrical appliance on
>(or, you have statistics supplied by the utilities....) you can't know.

I know when the power fails at home because I walk in and see a few
clocks blinking, the microwave oven, etc. I can tell how long the power
was off by how slow the clocks with hands are running. I know when
the power fails during the night in an office, because the net is down
the next morning, the system I left running all night rebooted, etc.
I have a odd friend who lives near a river, with such miserable service
that he has a battery system with inverters to keep his 11 PCs running.
(His hobby is weather prediction.) People make power line monitors to
detect even small outages. Dranetz (?)

>> and much better telephone service
>
>ditto

Yes, that's harder to tell, unless you use them a lot. 

>> the expected number of seconds a year of electrical outage,
>> when  neither  system is working, is .... 20 seconds
>
>this is fine as it stands, but it neglects one of the most important
>and troublesome aspects of reliability modelling - common cause
>(or common mode) failures. No matter how different your two systems
>may appear to be, they are not independent. At best, you could
>describe them as 'partially diverse'. There will be commom elements in
>both, such as contact breakers, fuses, even the wiring itself.

That's a deliberate neglection.

I think of this as a matter of degree. And some systems are easier and
cheaper to make redundant than others, and not uncoincidentally, more
reliable when made redundant that others. I'm thinking of power supplies
with diode-ORed outputs, or some other kind of paralleled outputs. Power
supplies are sometimes the most unreliable parts in a computer system,
and sometimes the items that can be most easily and reliably paralled. If
the switchover circutry were no more than two diodes, it would probably 
be a lot more reliable than the rest of a switching supply. 

I like to think about the marginal return on an investment in reliability:
where would you spend the first dollar, in making a system more reliable?
Where would you spend the next one? How hard is the switchover? How
fine-grained should the redundancy be? It's an interesting game to play
with a block diagram... And how does the cost of making a system more
reliable compare to the cost of a failure? I like to use numbers for this.

It seems to me that systems that handle significant power, eg AC inverters
(vs say, computers), often have costs that are proportional to their power-
handling capability, eg in their semiconductors and heat sinks and
transformers (vs. their cheap chip controllers), and easy switchover schemes
(compared to computers.) To me, that means that a redundant power supply
system which keeps working if N-1 out of N supplies work, may not cost a
lot more than one single larger supply, and it might be a lot more reliable.
It is easier to show this with numbers than words.

>The presence of common elements means that both systems could fail for the
>same reason. Take the example of two diesel generators, one running
>and one standby. This appears to be a redundant system, until you
>realise that they both require diesel oil, and this oil will very
>likely come from one source.

The Middle East?

>Dirty diesel oil will cause both generators to fail for the same reason,
>and at the same time (and this is a real example...). So, you cannot assume
>that all failures are independent.

I agree. And I'm not assuming that, but as a matter of degree, if things
can be arranged so that there are very few common elements compared to the
more complicated subsystems, with a much lower failure rate for the
common elements, then one is ahead of the game, however you would like
to describe the game in words. (Don't oil filters help?)

>Common cause failures are notoriously difficult to model. There is a
>whole branch of reliability modelling devoted to it.

I've seen some of that. Complicated, sometimes. 

>A simple way of modelling them is to say that a proportion of all failures
>in a system will be due to common causes. In a single system, of course, they
>wouldn't be recognised as such. Typical figures for this proportion
>are 20% for redundant systems, 6% for partially diverse systems, and
>2% for full diversity.

Interesting typical figures.

>In order to determine the unavailability due to common causes, this
>simple model allows us merely to take this proportion of the unavailability
>of a single system. In your example, the unavailability of a single system
>is 8E-4. The proportion of this that is due to common causes is 20%
>(for a redundant system)

It is?

>or an unavailability of 8E-4 x 0.2 = 1.6E-4, which equates to 1.4 hours
>per year. If we could argue partial diversity, then the systems would be
>unavailable for 25 minutes per year.

I'm thinking more about full diversity, with an attempt at clever design,
and picking the right things to diversify. 

>Of course, if the failures are coincident (cf the diesel generators)
>then the downtime is more likely to be the repair time, i.e. back to
>8 hours.

One would hope to avoid that in the design, no?

>Much more realistic, don't you think?

No.

>So, throwing lots of money at the problem in order to provide backup systems
>does not have the dramatic effect you suggested...

Aha, a conclusion :-)

Nick


References: