ZIL saturation

[Ian Collins]

Looking at the "pure SSD pool layout" thread made me wonder what
conditions will saturate one or two Intel 3700 SSDs when used as a ZIL?

It's easy enough on direct attached storage to generate enough
synchronous writes from a synthetic benchmark, but I haven't managed to
do so over NFS, even over 10GE.

I guess the question really comes down to how many write IOPs are
realistic over a 10GE link?

[jason matthews]

i accidentally tested an s3700 800gb device with sync writes on using filebench - my intention was sync off. if you are interested, i can retest to be sure but i want to say it generated about 3000 writes/second for 8k writes with sync using the "random write" test. i have no recollection of what the latency was.

j.

Looking at the "pure SSD pool layout" thread made me wonder what conditions will saturate one or two Intel 3700 SSDs when used as a ZIL?
It's easy enough on direct attached storage to generate enough synchronous writes from a synthetic benchmark, but I haven't managed to do so over NFS, even over 10GE.
I guess the question really comes down to how many write IOPs are realistic over a 10GE link?
--
Ian.
-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/22567878-8480fd5f
Modify Your Subscription: https://www.listbox.com/member/?&
Powered by Listbox: http://www.listbox.com

[Ian Collins]

Post by jason matthews
i accidentally tested an s3700 800gb device with sync writes on using filebench - my intention was sync off. if you are interested, i can retest to be sure but i want to say it generated about 3000 writes/second for 8k writes with sync using the "random write" test. i have no recollection of what the latency was.

Was that over NFS?

Either way, I'd be interested in the filebench options used.

[Richard Elling]

Looking at the "pure SSD pool layout" thread made me wonder what conditions will saturate one or two Intel 3700 SSDs when used as a ZIL?
It's easy enough on direct attached storage to generate enough synchronous writes from a synthetic benchmark, but I haven't managed to do so over NFS, even over 10GE.
I guess the question really comes down to how many write IOPs are realistic over a 10GE link?

10GbE wire speed is around 300k IOPS @ 4k. But it is easier to think in terms of bandwidth.
An Intel DC S3700 at 100GB is rated at 200MB/sec and a 10 GbE link is around 1GB/sec.
So in theory, you will need qty 5 (or 10 if you mirror) S3700s for your separate log.

I am not aware of a SAS or SATA SSD that claims 1GB/sec writes and is of the standard
2.5" or 3.5" form factor. For example, you need qty 2 (4 for mirror) of ZeusRAM to reach the
same level.

Note: assuming logbias=latency in the above analysis.
-- richard

--

***@RichardElling.com
[phone]












-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

[Saso Kiselkov]

Post by Richard Elling

Post by Ian Collins
Looking at the "pure SSD pool layout" thread made me wonder what
conditions will saturate one or two Intel 3700 SSDs when used as a ZIL?
It's easy enough on direct attached storage to generate enough
synchronous writes from a synthetic benchmark, but I haven't managed
to do so over NFS, even over 10GE.
I guess the question really comes down to how many write IOPs are
realistic over a 10GE link?

An Intel DC S3700 at 100GB is rated at 200MB/sec and a 10 GbE link is around 1GB/sec.
So in theory, you will need qty 5 (or 10 if you mirror) S3700s for your separate log.
I am not aware of a SAS or SATA SSD that claims 1GB/sec writes and is of the standard
2.5" or 3.5" form factor. For example, you need qty 2 (4 for mirror) of
ZeusRAM to reach the
same level.
Note: assuming logbias=latency in the above analysis.

Keep in mind that the DC S3700 has different chip counts depending on
capacity and thus also throughput. It might make sense to look into
200GB or 400GB models, which IIRC, have significantly higher performance
numbers (both in IOPS as well as raw throughput).

Cheers,

[Andrew Galloway]

Also might be worth noting that how ZIL I/O looks to a log device is a bit
different than what a typical benchmark application is going to even be
capable of generating.

Post by Saso Kiselkov

Post by Richard Elling

Post by Ian Collins
Looking at the "pure SSD pool layout" thread made me wonder what
conditions will saturate one or two Intel 3700 SSDs when used as a ZIL?
It's easy enough on direct attached storage to generate enough
synchronous writes from a synthetic benchmark, but I haven't managed
to do so over NFS, even over 10GE.
I guess the question really comes down to how many write IOPs are
realistic over a 10GE link?

terms of bandwidth.
An Intel DC S3700 at 100GB is rated at 200MB/sec and a 10 GbE link is around 1GB/sec.
So in theory, you will need qty 5 (or 10 if you mirror) S3700s for your separate log.
I am not aware of a SAS or SATA SSD that claims 1GB/sec writes and is of the standard
2.5" or 3.5" form factor. For example, you need qty 2 (4 for mirror) of
ZeusRAM to reach the
same level.
Note: assuming logbias=latency in the above analysis.

Keep in mind that the DC S3700 has different chip counts depending on
capacity and thus also throughput. It might make sense to look into
200GB or 400GB models, which IIRC, have significantly higher performance
numbers (both in IOPS as well as raw throughput).
Cheers,
--
Saso
-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
https://www.listbox.com/member/archive/rss/182191/24484421-62d25f20
https://www.listbox.com/member/?&
Powered by Listbox: http://www.listbox.com

-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

[jason matthews]

Also might be worth noting that how ZIL I/O looks to a log device is a bit different than what a typical benchmark application is going to even be capable of generating.

<engineer_speak>Synthetic benchmarks are always, well, synthetic but they give us something to talk about in relative terms. These terms may or may have relevance to a particular application.</engineer_speak>


j.


-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

[Keith Wesolowski]

Post by jason matthews

Also might be worth noting that how ZIL I/O looks to a log device is a bit different than what a typical benchmark application is going to even be capable of generating.

<engineer_speak>Synthetic benchmarks are always, well, synthetic but they give us something to talk about in relative terms. These terms may or may have relevance to a particular application.</engineer_speak>

Fortunately in this case it's relatively easy to test what you care
about. Create a pool using the device you want to evaluate as a slog,
then use a filesystem-based benchmark (synthetic or real) and measure
delivered performance. Use DTrace to observe the behaviour of the slog
under various filesystem workloads to determine where it is or is not
the limiting factor, what the device-level workload looks like, and how
the device itself is performing. Device-only synthetic benchmarks are
completely useless for slogs, and there's no reason to bother with them.

[Sam Zaydel]

Another important point here about synthetic IO generation tools is that
you may in fact create a profile that does nothing to exercise the ZIL and
the slog device if you have one. This is where it might be well worthwhile
actually testing with something more realistic or with a profile that you
know for a fact will take correct path through the system, or by setting
sync=always on the dataset against which you are testing.


On Thu, Nov 14, 2013 at 10:00 AM, Keith Wesolowski <

Post by jason matthews

Post by Andrew Galloway
Also might be worth noting that how ZIL I/O looks to a log device is a

bit different than what a typical benchmark application is going to even be
capable of generating.

Post by jason matthews
<engineer_speak>Synthetic benchmarks are always, well, synthetic but

they give us something to talk about in relative terms. These terms may or
may have relevance to a particular application.</engineer_speak>
Fortunately in this case it's relatively easy to test what you care
about. Create a pool using the device you want to evaluate as a slog,
then use a filesystem-based benchmark (synthetic or real) and measure
delivered performance. Use DTrace to observe the behaviour of the slog
under various filesystem workloads to determine where it is or is not
the limiting factor, what the device-level workload looks like, and how
the device itself is performing. Device-only synthetic benchmarks are
completely useless for slogs, and there's no reason to bother with them.
-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
https://www.listbox.com/member/archive/rss/182191/24342081-7731472e
https://www.listbox.com/member/?&
Powered by Listbox: http://www.listbox.com

--
Join the geek side, we have π!

Please feel free to connect with me on LinkedIn.
http://www.linkedin.com/in/samzaydel



-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com