Subject: Re: Second level cache?

From: Alan Krantz <[email-masked]>
Date: Mon, 9 Aug 93 09:13:39 MDT

> From [email-masked] Mon Aug  9 08:53:13 1993
  > Status: R

  > Somebody mentioned that some research needs to be done on how a second level 
  > cache will effect performance.  Any suggestions on how to do this?

  > Based on observation it seems that a cache is worth it, SGI mips 4000 SC is
  > around 35/34 and the mips 4000 PC is around 60/58 specint/specfp.

  > Adding the decreased locality of Unix+X11 over dos, the higher (100 Mhz
  > internal) clock rate, and mips binaries being bigger then 486 it would
  > seem that a large secondary cache would be necessary.  Based on a 486's
  > secondary cache being useful.

The subject isn't that simple. For example the usefulness of the cache 
greately depends on how the memory is interleaved (the R4xxx has an external
64 bit path way). Also there is an issue of the internal cache size. An
internal cache on a 486 is 16k (I think 8k each for I&D) (but writethrough not
write back). The R4xxxx has a larger internal cache.

  > Any suggestions on how to get some numbers experimentally?

Yea build an R4000 system with the desired level of interleaving; one with
and one without secondary cache. Then run both systems :)

---

From: [email-masked]
Date: Mon, 09 Aug 93 17:23:56 +0200

You write: 

> and one without secondary cache. Then run both systems :)

Ok, I'll throw in my ballot for the cache one...

Can I "test-drive" for free?  ;-)

I don't know much about hardware design, but could you design the thing
in such a way that you can plug in cache, when you need it.  IE: if you
don't have the money, the board will run without the SC, and if you do,
you just buy some SRAM, and plug 'er in...

Or will this cause the logic to go beserk?

--Toby.
--------------------------------------------------------------
|Tobias Weingartner  |    PGP2.x Public Key available at     |
| +41'01'254'7205    |   'finger [email-masked]'   |
--------------------------------------------------------------
%SYSTEM-F-ANARCHISM, the operating system has been overthrown

---

From: Drew Eckhardt <[email-masked]>
Date: Mon, 09 Aug 1993 21:56:57 -0600

The subject isn't that simple. For example the usefulness of the cache 
    greately depends on how the memory is interleaved 

Yes.  2-way interleaving can cut the wait states in half in the 
average case with a cache-miss, 4-way in half again, etc.  However,
the 4000, 4200, and 4400 are all 64 bit chips meaning you need 
a 128 bit wide memory subsystem for 2-way, 256 for 4-way.
    
    Also there is an issue of the internal cache size. An
    internal cache on a 486 is 16k 

No.  The internal cache on an Intel 486 is 8K combined I&D, although
IBM is building it's own 486's that may have a larger internal cache.

    but writethrough not write back. 

Yes.  This is very significant.  Also important are cache interleave,
line size, etc.  

    The R4xxxx has a larger internal cache.

R4000 : 8K I, 8K D
R4400 : 16K I, 16K D

The maximum imposed by the R4000 architecture is 32K each for the primary
cache.  I don't know about the 4200 since I don't have a data book in front
of me like I do for the NEC Vr4000 and Vr4400, PC, SC, and MC variants :-)

Yep.
    
--------

---

From: Drew Eckhardt <[email-masked]>
Date: Mon, 09 Aug 1993 22:08:21 -0600

It could be configurable (ie, my i386 uses jumpers), the question is 
can a second level cache be done within the constraints of our
cost design goal.

R4x00 SC and MC chips include support for a second level cache, ie you 
throw enough SRAM on for tag, tag ECC, data, and data ECC with a 
few PALs for address decode and have it.  However, preliminary reports 
indicate the 4000SC has a ~$600 price tag rather than ~$300 for the 
4000PC, meaning using an SC is not practical in our design entirely
because of the cost constraint.

The question becomes : can we meet our cost target by having 
a second level cache *external* to the processor, with low
cost boards shipped with 0K SRAM.

--------

---

From: [email-masked]
Date: Tue, 10 Aug 93 09:29:38 +0200

You write: 

> can a second level cache be done within the constraints of our

What is our cost design goal?  $2000 US?  Less?  More?

> because of the cost constraint.

For low cost, I would not think about the SC/MC chips.  They have
400+ pins, and that will be a nightmare to make a cheap board out of.

I was more thinking along the lines of a 4000PC, with an external
cache controller.  I don't know if it's possible, but that would
take our pin count down to ~180 pins...

--Toby.

---

From: Drew Eckhardt <[email-masked]>
Date: Tue, 10 Aug 1993 05:06:32 -0600

From what I remember, discussions on the mailing list have placed this 
somewhere between $600 and $800 for a mainboard with dumb frame 
buffer (using MAC VRAM SIMMs), SCSI, serial ports, parallel
ports, IDE, and ethernet onboard, where some unpopular 
features (ethernet has been named as the first thing to
go in the base configuration) may be designed in but the 
sockets left empty to facilitate meeting this goal.

That's about what I was getting at.

I had been thinking along the same lines, but don't know 
how practical it would be.

Existing cache sets aren't likely to directly support the r4000 system
interface, (64 bit A/D bus + 8 bit parity / ECC + 9 bit command 
bus +  1 bit parity + handshaking signals) since this is a departure
from other systems, the chips are new, and the high end chips that
will be using external caches allready support them.

Weather we can force something existing to do what we want is 
another issue - you can ignore the high 32 bits of the address, you 
can disable parity/ECC checking, can mangle various read and write 
commands to look like what the external cache controller expects,
the 4000 chips flag the last read/write in a burst transfer so bursts
could be handled with counter latched when the address was specified, 
incrementing on each read/write handshake, etc. but the external logic to 
do all of this will increase propogation delays which might make it 
impossible with the parts when can afford.

Another alternative would be be having what passes as a memory
controller in the talk to some other cache/memory controller
instead of DRAM, again, you can't say that it will work until
you see what the timing looks like.

IMHO : a lot of things would be "neat," ie the second level
cache, but no one has suggested chips that would be useful, with 
pricing for said chips to determine weather they work for 
us.

Waiting for an initial lead on these chips to drop out of thin air
(as was the case with the 4200 with someone stumbling across a MIPS 
press on the net, other people following up on NEC second sourcing
them, and people on the list finally ending up with more solid data
some time later) will take a random, potentially infinite amount
of time :-)

IMHO : We need to get parts and prices so their feasibility can be studied
before a design is done, or we need to go ahead with a design so us
software people have something to play with.

    --Toby.

--------

---

From: [email-masked]
Date: Tue, 10 Aug 93 15:05:22 +0200

You write: 

> sockets left empty to facilitate meeting this goal.

Ok, junk the IDE.  I realize that you can get cheap IDE drives, etc,
but SCSI is getting pretty cheap too, and fast SCSI would be pleanty fast
for a *personal* machine.  Then again, if it only costs a cent or two,
why not keep it in.  Make the PC dudes happy.

I'd definitely want to be able to use ethernet.  Wether that will be add
on, or not is a money issue (I persume).  The machine becomes next to
unusable to me if I have to resort to SLIP over a slow link...

(What type of ethernet connector would this be?)

> will be using external caches allready support them.

What sort of bandwidth does the R4kPC need?  How about some real
figures, using from 1 - N waitstates, from 0 - 100 MHz, etc.  I don't
have a databook handy, so can someone look them up?

> impossible with the parts when can afford.

Hmm, does MIPS (or an alternative manufacturer) offer a DRAM controller
that directly interfaces to the R4kPC?  That might be the cheapest
(maybe fastest too) way to go.

Maybe a more cost effective alternative (already mentioned once) would be
to just interleave the DRAM.  Not as "neat" as a cache, but if we need to
keep cost down below US$800...

> software people have something to play with.

I agree with the first version of this.  This way the design will not change
once it is laid down.  That way the software does not have to be kludged to
fit later.  From working on PC systems, I find a lack of percise (sp?) and
clear documentation to be the biggest stumbling block.  IE: How can you port
an OS, if you don't have the specs to the machine?

--Toby.

---

From: [email-masked] (Keith Rohrer)
Date: Tue, 10 Aug 1993 08:48:23 -0600 (CDT)

[as elm fries attributions]
> Ok, junk the IDE.  I realize that you can get cheap IDE drives, etc,
Junk IDE as built-in to the mainboard, but don't forget the ISA bus!
Unless, as is said later, the IDE-on-the-motherboard is dirt cheap,
nobody really needs it; on the other hand, for expansion, lots of
us want or need ISA...

> but SCSI is getting pretty cheap too, and fast SCSI would be pleanty fast
For me, the cost of SCSI would be the cost of replacing ~450 megabytes of
hard drive.  Not happening.  Sorry.
> why not keep it in.  Make the PC dudes happy.
You mean, don't invalidate some people's current investment?

[deletia]
 
	Keith

---

From: [email-masked] (Neil Russell)
Date: Tue, 10 Aug 93 11:16:25 PDT

> you see what the timing looks like.

What I understand is that the ARC chipset supports a secondary cache
similar to the way it does for the 486.  It is not necessary to use
the cache support signals from the R4k to have a secondary cache, although
it may mean that this cache will take two cycles to access data where
the R4000SC cache would take one.  This may not be great, but it is still
better than the cycles required to get data from DRAM.

Note that I am guessing about the timing issue; it may be that the ARC
chipset's cache logic can handle one cycle access.

With the ARC chipset's arangement, it is possible to add cache the same
as you would with a 486 MB; probably using the same cache chips.

BTW, Pyramid has a R4000MC machine, which uses the maximum 4Mbytes of
secondary cache.  I will ask if anyone has done some serious benchmarks
on how well this works.

> software people have something to play with.

I, like quite a few others on this list, am hassling a distributor about
the NEC r4k.  I am having just as much joy as everyone else - none... ;-(
My personal opinion is that the R4k + ARC is the best solution that we
have.  Unfortunately, we cannot do anything without that information.
Stay tuned; when the manufacturers are forthcomming with the data,
we will be forthcomming with a board.

-- 
Neil Russell		(The wizard from OZ)
Pyramid Technology			Email:  [email-masked]
3860 N. First Street			Voice:  (408) 428-7302
San Jose, CA 95134-1702			  FAX:  (408) 428-8845

---

From: Tommy Thorn <[email-masked]>
Date: Wed, 11 Aug 1993 12:02:49 +0200

Andreas Busse writes:
 > 5. The R4200 ***DOES NOT*** support 2nd level cache. The Blockdiagram
 >    of ARCset shows no 2nd level cache, and there's ***NO SUPPORT*** at all.

Huh? 

 > Finally, I simply don't understand why this discussion came up.
 > I thought our goal was to design and produce a board with very good
 > price/performance relation. Although the price of the ARCset isn't
 > clear at the moment (which is not my or our fault, but NEC's) it seems
 > that this is what we are looking for. It offers more speed than
 > a R3081 solution probably for only few $s more.

I suppose the discussion came becourse there is evidence that cache
size can influence performens (big news ;^) It is then a natural
question to ask "Can we have more? How much will it cost?" That the
ARCset didn't support it was/is non-obvious for the most of us. (Given
that most of us don't have any info on the ARCset besides what's
revealed in this forum.)

 > If you all insist on a 2nd level cache design, we should forget
 > about the ARCset. That means in turn that we should forget about
 > having a design within the next 12 months. I'm not sure what then
 > happens to the group...

Given this, we proberly don't wan't 2nd level cache, but don't flame
us for having the discussion.

/Tommy Thorn