Finding the "last" snapshot for a zfs volume

Discussion:

Jonathan Adams via illumos-zfs

2014-05-14 08:54:08 UTC

I've posted this to the OpenIndiana-discuss list, and they suggested that I
pass it on to you.

--

On our backup servers I have a "backup-check" script that checks to see
what the last snapshot for a particular volume is (and compares it with the
current date)

df -F zfs -h |grep $mypool | nawk '{system("zfs list -t snapshot -r "$1" \|
tail -1")}' | egrep -v "$yesterday|no datasets"

okay, it's messy, but it should find the last snapshot for each zfs slice
within a pool.

However with the number of pools on the server increasing, and the number
of snapshots getting bigger every day, I was wondering if there is an
easier way of getting the latest snapshot than "zfs list -t snapshot -r
pool/snapshot | tail -1"

Any ideas would be helpful.

--

All the suggestions involved either "zfs list -t snapshot > tempfile"
followed greps and tails, or "zfs list -H -d 1 -o name -S creation -t
snapshot "$target" | head -n 1" equivalents.

What I'm trying to do is find out if a particular snapshot exists.
pool/slice/***@yesterdays-formatted-date ... is there any way of
quickly finding out if this exists?

Thanks in advance.

Jonathan Adams

-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

Fajar A. Nugraha via illumos-zfs

2014-05-14 09:07:29 UTC

Permalink

On Wed, May 14, 2014 at 3:54 PM, Jonathan Adams via illumos-zfs <

Post by Jonathan Adams via illumos-zfs
What I'm trying to do is find out if a particular snapshot exists.
quickly finding out if this exists?

Do you mean something like this?

# zfs list -o name -H -r -t snapshot data/nfs | grep
@zfs-auto-snap_daily-$(date -d yesterday +%Y-%m-%d-)
data/***@zfs-auto-snap_daily-2014-05-13-2349

It will return error code "1" if grep can't find matching line (and grep
obviously won't return any output). Note that I'm using this on linux, with
date from GNU coreutils. Not sure if solaris/bsd's date have the same
feature.
--
Fajar

-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

Jonathan Adams via illumos-zfs

2014-05-14 09:11:55 UTC

Permalink

I was wondering if there was a way of searching for a snapshot without
waiting for the pool to wake up, scroll through the list of 1000+ snapshots
for a specific slice and find out if it received last nights backup ... for
12 pools with ~ 15 slices per pool.

When I run the "backup-check" script it can take over 1/2 hour to return
...

It'd be nice to just say either:
1) what is the last snapshot on this slice?
or
2) does a specific snapshot exist?

Jon

Post by Fajar A. Nugraha via illumos-zfs
On Wed, May 14, 2014 at 3:54 PM, Jonathan Adams via illumos-zfs <

Post by Jonathan Adams via illumos-zfs
What I'm trying to do is find out if a particular snapshot exists.
quickly finding out if this exists?

Do you mean something like this?
# zfs list -o name -H -r -t snapshot data/nfs | grep
@zfs-auto-snap_daily-$(date -d yesterday +%Y-%m-%d-)
It will return error code "1" if grep can't find matching line (and grep
obviously won't return any output). Note that I'm using this on linux, with
date from GNU coreutils. Not sure if solaris/bsd's date have the same
feature.
--
Fajar

-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

Fajar A. Nugraha via illumos-zfs

2014-05-14 09:20:31 UTC

Permalink

Post by Jonathan Adams via illumos-zfs
1) what is the last snapshot on this slice?

I don't think there's any way

or

Post by Jonathan Adams via illumos-zfs
2) does a specific snapshot exist?

Did you try "zfs list ***@snapshot_name"? It should return error code 1
if the specified snapshot does not exist. In my case I can't use this
method because zfs-auto-snapshot script adds hour and minute to the
snapshot name.

My method earlier (zfs list -t snapshot -r) will only list snapshots for
that dataset only, so it should be much faster compared to listing all
snapshots.
--
Fajar

-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

Jonathan Adams via illumos-zfs

2014-05-14 09:48:43 UTC

Permalink

Post by Jonathan Adams via illumos-zfs
I was wondering if there was a way of searching for a snapshot without
waiting for the pool to wake up, scroll through the list of 1000+ snapshots
for a specific slice and find out if it received last nights backup ... for
12 pools with ~ 15 slices per pool.
When I run the "backup-check" script it can take over 1/2 hour to return

That doesn't sound quite right. Is this machine RAM starved?

[***@mansalbck01] ~# dmesg | grep memory
real memory = 68719476736 (65536 MB)
avail memory = 66422546432 (63345 MB)

[***@mansalbck01] ~# dmesg | grep CPU
CPU: Intel(R) Core(TM) i7-3820 CPU @ 3.60GHz (3602.13-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs

[***@mansalbck01] ~# swapinfo -h
Device 1K-blocks Used Avail Capacity
/dev/da16p1.eli 2097152 0B 2.0G 0%
/dev/da7p1.eli 2097152 0B 2.0G 0%
/dev/da4p1.eli 2097152 0B 2.0G 0%
/dev/da13p1.eli 2097152 0B 2.0G 0%
/dev/da17p1.eli 2097152 0B 2.0G 0%
/dev/da6p1.eli 2097152 0B 2.0G 0%
/dev/da5p1.eli 2097152 0B 2.0G 0%
/dev/da12p1.eli 2097152 0B 2.0G 0%
/dev/da18p1.eli 2097152 0B 2.0G 0%
/dev/da3p1.eli 2097152 0B 2.0G 0%
/dev/da2p1.eli 2097152 0B 2.0G 0%
/dev/da15p1.eli 2097152 0B 2.0G 0%
/dev/da1p1.eli 2097152 0B 2.0G 0%
/dev/da20p1.eli 2097152 0B 2.0G 0%
/dev/da0p1.eli 2097152 0B 2.0G 0%
/dev/da9p1.eli 2097152 0B 2.0G 0%
/dev/da22p1.eli 2097152 0B 2.0G 0%
/dev/da8p1.eli 2097152 0B 2.0G 0%
/dev/da21p1.eli 2097152 0B 2.0G 0%
/dev/da10p1.eli 2097152 0B 2.0G 0%
/dev/da14p1.eli 2097152 0B 2.0G 0%
/dev/da19p1.eli 2097152 0B 2.0G 0%
Total 46137344 0B 44G 0%

no, it's not ram starved.

actually just tried it now, and it came back in ~ 1 minute ... must have
been busy/asleep yesterday.

It'll return 0 if the snapshot exists, and 1 if it doesn't.

[***@mansalbck01] ~# df -t zfs -h |grep ptserver | nawk '{system("zfs
list -H "$1"@'$yesterday'")}'

came back instantly ... that might well be worth it ... If I can get it to
return a value, and only look for the last line if it doesn't exist then I
should be able to speed up the command instantly.

Thanks

Jon

-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com

Simon Casady via illumos-zfs

2014-05-14 12:13:23 UTC

Permalink

If snapdir=visible then you can use ls and friends but I suspect it would
not be any faster.

On Wed, May 14, 2014 at 4:48 AM, Jonathan Adams via illumos-zfs <

Post by Jonathan Adams via illumos-zfs

That doesn't sound quite right. Is this machine RAM starved?

real memory = 68719476736 (65536 MB)
avail memory = 66422546432 (63345 MB)
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
Device 1K-blocks Used Avail Capacity
/dev/da16p1.eli 2097152 0B 2.0G 0%
/dev/da7p1.eli 2097152 0B 2.0G 0%
/dev/da4p1.eli 2097152 0B 2.0G 0%
/dev/da13p1.eli 2097152 0B 2.0G 0%
/dev/da17p1.eli 2097152 0B 2.0G 0%
/dev/da6p1.eli 2097152 0B 2.0G 0%
/dev/da5p1.eli 2097152 0B 2.0G 0%
/dev/da12p1.eli 2097152 0B 2.0G 0%
/dev/da18p1.eli 2097152 0B 2.0G 0%
/dev/da3p1.eli 2097152 0B 2.0G 0%
/dev/da2p1.eli 2097152 0B 2.0G 0%
/dev/da15p1.eli 2097152 0B 2.0G 0%
/dev/da1p1.eli 2097152 0B 2.0G 0%
/dev/da20p1.eli 2097152 0B 2.0G 0%
/dev/da0p1.eli 2097152 0B 2.0G 0%
/dev/da9p1.eli 2097152 0B 2.0G 0%
/dev/da22p1.eli 2097152 0B 2.0G 0%
/dev/da8p1.eli 2097152 0B 2.0G 0%
/dev/da21p1.eli 2097152 0B 2.0G 0%
/dev/da10p1.eli 2097152 0B 2.0G 0%
/dev/da14p1.eli 2097152 0B 2.0G 0%
/dev/da19p1.eli 2097152 0B 2.0G 0%
Total 46137344 0B 44G 0%
no, it's not ram starved.
actually just tried it now, and it came back in ~ 1 minute ... must have
been busy/asleep yesterday.

It'll return 0 if the snapshot exists, and 1 if it doesn't.

came back instantly ... that might well be worth it ... If I can get it to
return a value, and only look for the last line if it doesn't exist then I
should be able to speed up the command instantly.
Thanks
Jon
*illumos-zfs* | Archives<https://www.listbox.com/member/archive/182191/=now>
<https://www.listbox.com/member/archive/rss/182191/24018577-4d8b86e0> |
Modify<https://www.listbox.com/member/?&>Your Subscription
<http://www.listbox.com>

Alex Waite via illumos-zfs

2014-05-14 09:20:40 UTC

Permalink

That doesn't sound quite right. Is this machine RAM starved?

That being said, I've never worked on a box with that many pools before.

Post by Jonathan Adams via illumos-zfs
...
1) what is the last snapshot on this slice?
or
2) does a specific snapshot exist?

I'm not aware of a way to achieve 1) without using the tail -1 method.
However, if the snapshot names are predictable (/exact/ same time every
day), then something as simple as this should work:

zfs list -H jackknife@${snapshot_name}

It'll return 0 if the snapshot exists, and 1 if it doesn't.

No need for `grep` or non-portable GNU/BSD specific `date` features.

I hope this helps.

---Alex

Matthew Ahrens via illumos-zfs

2014-05-14 12:56:32 UTC

Permalink

We need to upstream our "prevsnap" property, it does exactly what you want.
Or if someone else wants to, feel free:

https://github.com/delphix/delphix-os/commit/95378d0bc82aeec9c2f7833f569cffcb86aa3c63

--matt

On Wed, May 14, 2014 at 1:54 AM, Jonathan Adams via illumos-zfs <

Post by Jonathan Adams via illumos-zfs
I've posted this to the OpenIndiana-discuss list, and they suggested that
I pass it on to you.
--
On our backup servers I have a "backup-check" script that checks to see
what the last snapshot for a particular volume is (and compares it with the
current date)
df -F zfs -h |grep $mypool | nawk '{system("zfs list -t snapshot -r "$1" \|
tail -1")}' | egrep -v "$yesterday|no datasets"
okay, it's messy, but it should find the last snapshot for each zfs slice
within a pool.
However with the number of pools on the server increasing, and the number
of snapshots getting bigger every day, I was wondering if there is an
easier way of getting the latest snapshot than "zfs list -t snapshot -r
pool/snapshot | tail -1"
Any ideas would be helpful.
--
All the suggestions involved either "zfs list -t snapshot > tempfile"
followed greps and tails, or "zfs list -H -d 1 -o name -S creation -t
snapshot "$target" | head -n 1" equivalents.
What I'm trying to do is find out if a particular snapshot exists.
quickly finding out if this exists?
Thanks in advance.
Jonathan Adams
*illumos-zfs* | Archives<https://www.listbox.com/member/archive/182191/=now>
<https://www.listbox.com/member/archive/rss/182191/21635000-ebd1d460> |
Modify<https://www.listbox.com/member/?&>Your Subscription
<http://www.listbox.com>

Jim Klimov via illumos-zfs

2014-05-14 17:04:38 UTC

Permalink

Hi all,
Here is my take at this problem:

# ( zfs list -Honame -tall -r rpool; echo "") | ggrep -B1 -v @ | grep @
(edit to omit "-r" or point it to the part of the tree you're interested in, or add "-t" to specify dataset types like "-t snapshot,volume").
This code depends on "zfs list" showing all snapshots of a dataset in calendar order and before recursing into children (IIRC older zfs could do this in another order), and on a grep with -B support. Effectively this finds all dataset names (no "@") and prints the line before that (another dataset without snapshots, or the last snapshot of the previous dataset), and then selects only snapshots (with "@"). This does only one "zfs" pass, each of which may be quite a time-consumer so you don't want to stack those ;)
HTH,
//Jim Klimov

-------------------------------------------
illumos-zfs
Archives: https://www.listbox.com/member/archive/182191/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182191/23047029-187a0c8d
Modify Your Subscription: https://www.listbox.com/member/?member_id=23047029&id_secret=23047029-2e85923f
Powered by Listbox: http://www.listbox.com