注意

本文档适用于 Ceph 开发版本。

Dumpling

Dumpling 是 Ceph 的第 4 个稳定版本。它以乌贼（Euprymna tasmanica）命名。

v0.67.12 “Dumpling” (草稿)

这个 Dumpling 的稳定更新修复了 OSD 中长期存在的问题，这些问题可能会导致 IO 停滞。此外，还修复了启用缓存时 librbd 中读取的内存利用率问题，以及系统其余部分的几个小修复。

在 backfill 期间遇到 IO 停滞的 Dumpling 用户，如果他们不久后不打算升级到 Firefly，则应该升级。其他人应该已经升级到 Firefly 了。这可能是 0.67.x Dumpling 系列的最后一个稳定版本。

值得注意的变化

  • buffer: 修复 buffer 重建对齐的边缘情况 (#6614 #6003 Loic Dachary, Samuel Just)

  • ceph-disk: 在 zap 后重新探测分区 (#9665 #9721 Loic Dachary)

  • ceph-disk: 在适当的时候使用 partx 而不是 partprobe (Loic Dachary)

  • common: 添加 $cctid 元变量 (#6228 Adam Crume)

  • crush: 修复 get_full_location_ordered (Sage Weil)

  • crush: 选择与 rule_id 匹配的规则集 id (#9675 Xiaoxi Chen)

  • libcephfs: 修复 tid 回绕错误 (#9869 Greg Farnum)

  • libcephfs: 在 -1 上获取 osd 位置应该返回 EINVAL (Sage Weil)

  • librados: 修复与 C API 和 op 超时相关的竞争条件 (#9582 Sage Weil)

  • librbd: 限制正在进行的读取请求的最大数量 (#9854 Jason Dillaman)

  • librbd: 在读取请求上执行缓存大小 (Jason Dillaman)

  • librbd: 修复图像打开失败路径中的无效关闭 (#10030 Jason Dillaman)

  • librbd: 修复稀疏文件上的读取挂起 (Jason Dillaman)

  • librbd: 优雅地处理已删除/重命名的池 (#10270 #10122 Jason Dillaman)

  • librbd: 保护 list_children 免受无效子池 ioctxs 的影响 (#10123 Jason Dillaman)

  • mds: 修复来自没有脏功能的客户端的 ctime 更新 (#9514 Greg Farnum)

  • mds: 修复 cap 更新路径中的罕见空指针解引用 (Greg Farnum)

  • mds: 修复系统时钟反向导致的断言 (#11053 Yan, Zheng)

  • mds: 在 straydir 上存储回溯 (Yan, Zheng)

  • osd: 修复重播后 journal committed_thru 更新的错误 (#6756 Samuel Just)

  • osd: 修复内存泄漏，忙循环在快照修剪上 (#9113 Samuel Just)

  • osd: 修复一些 peering 和恢复错误 (#10168 Samuel Just)

  • osd: 修复 backfill 开始时的 purged_snap 字段 (#9487 Sage Weil, Samuel Just)

  • osd: 处理无操作写入的快照边缘情况 (#10262 Sage Weil, Loic Dachary)

  • osd: 尊重 RWORDERED rados 标志 (Sage Weil)

  • osd: 一些 backfill 修复和重构 (Samuel Just, David Zafman)

  • rgw: 在 fastcgi 中显式发送 http 状态原因 (Yehuda Sadeh)

v0.67.11 “Dumpling”

这个 Dumpling 的稳定更新修复了影响一小部分用户的几个重要错误。

我们建议所有 Dumpling 用户在其方便的时候升级。如果这些问题中没有一个是影响您的部署，则没有紧迫性。

值得注意的变化

  • common: 修复发送重复的集群日志项 (#9080 Sage Weil)

  • 文档: 一些文档更新 (Alfredo Deza)

  • libcephfs-java: 修复与旧 JNI 头文件构建的构建问题 (Greg Farnum)

  • librados: 修复 op 超时路径中的崩溃 (#9362 Matthias Kiefer, Sage Weil)

  • librbd: 修复使用扁平化图像克隆时崩溃 (#8845 Josh Durgin)

  • librbd: 修复打开图像时错误路径清理 (#8912 Josh Durgin)

  • mon: 在添加任何 OSD 之前调整 pg_num 时修复崩溃 (#9052 Sage Weil)

  • mon: 减少来自 Paxos 的日志噪音 (Aanchal Agrawal, Sage Weil)

  • osd: 允许 scrub 和快照修剪线程池 IO 优先级被调整 (Sage Weil)

  • osd: 修复挂载/重新挂载同步竞争 (#9144 Sage Weil)

v0.67.10 “Dumpling”

This stable update release for Dumpling includes primarily fixes for RGW, including several issues with bucket listings and a potential data corruption problem when multiple multi-part uploads race. There is also some throttling capability added in the OSD for scrub that can mitigate the performance impact on production clusters.

We recommend that all Dumpling users upgrade at their convenience.

值得注意的变化

  • ceph-disk: partprobe befoere settle, fixing dm-crypt (#6966, Eric Eastman)

  • librbd: add invalidate cache interface (Josh Durgin)

  • librbd: close image if remove_child fails (Ilya Dryomov)

  • librbd: fix potential null pointer dereference (Danny Al-Gaaf)

  • librbd: improve writeback checks, performance (Haomai Wang)

  • librbd: skip zeroes when copying image (#6257, Josh Durgin)

  • mon: fix rule(set) check on ‘ceph pool set … crush_ruleset …’ (#8599, John Spray)

  • mon: shut down if mon is removed from cluster (#6789, Joao Eduardo Luis)

  • osd: fix filestore perf reports to mon (Sage Weil)

  • osd: force any new or updated xattr into leveldb if E2BIG from XFS (#7779, Sage Weil)

  • osd: lock snapdir object during write to fix race with backfill (Samuel Just)

  • osd: option sleep during scrub (Sage Weil)

  • osd: set io priority on scrub and snap trim threads (Sage Weil)

  • osd: ‘status’ admin socket command (Sage Weil)

  • rbd: tolerate missing NULL terminator on block_name_prefix (#7577, Dan Mick)

  • rgw: calculate user manifest (#8169, Yehuda Sadeh)

  • rgw: fix abort on chunk read error, avoid using extra memory (#8289, Yehuda Sadeh)

  • rgw: fix buffer overflow on bucket instance id (#8608, Yehuda Sadeh)

  • rgw: fix crash in swift CORS preflight request (#8586, Yehuda Sadeh)

  • rgw: fix implicit removal of old objects on object creation (#8972, Patrycja Szablowska, Yehuda Sadeh)

  • rgw: fix MaxKeys in bucket listing (Yehuda Sadeh)

  • rgw: fix race with multiple updates to a single multipart object (#8269, Yehuda Sadeh)

  • rgw: improve bucket listing with delimiter (Yehuda Sadeh)

  • rgw: include NextMarker in bucket listing (#8858, Yehuda Sadeh)

  • rgw: return error early on non-existent bucket (#7064, Yehuda Sadeh)

  • rgw: set truncation flag correctly in bucket listing (Yehuda Sadeh)

  • sysvinit: continue starting daemons after pre-mount error (#8554, Sage Weil)

更多详细信息，请参阅the complete changelog.

v0.67.9 “Dumpling”

This Dumpling point release fixes several minor bugs. The most prevalent in the field is one that occasionally prevents OSDs from starting on recently created clusters.

We recommend that all Dumpling users upgrade at their convenience.

值得注意的变化

  • ceph-fuse, libcephfs: client admin socket command to kick and inspect MDS sessions (#8021, Zheng Yan)

  • monclient: fix failure detection during mon handshake (#8278, Sage Weil)

  • mon: set tid on no-op PGStatsAck messages (#8280, Sage Weil)

  • msgr: fix a rare bug with connection negotiation between OSDs (Guang Yang)

  • osd: allow snap trim throttling with simple delay (#6278, Sage Weil)

  • osd: check for splitting when processing recover/backfill reservations (#6565, Samuel Just)

  • osd: fix backfill position tracking (#8162, Samuel Just)

  • osd: fix bug in backfill stats (Samuel Just)

  • osd: fix bug preventing OSD startup for infant clusters (#8162, Greg Farnum)

  • osd: fix rare PG resurrection race causing an incomplete PG (#7740, Samuel Just)

  • osd: only complete replicas count toward min_size (#7805, Samuel Just)

  • rgw: allow setting ACLs with empty owner (#6892, Yehuda Sadeh)

  • rgw: send user manifest header field (#8170, Yehuda Sadeh)

更多详细信息，请参阅the complete changelog.

v0.67.8 “Dumpling”

This Dumpling point release fixes several non-critical issues since v0.67.7. The most notable bug fixes are an auth fix in librbd (observed as an occasional crash from KVM), an improvement in the network failure detection with the monitor, and several hard to hit OSD crashes or hangs.

We recommend that all users upgrade at their convenience.

升级

  • The ‘rbd ls’ function now returns success and returns an empty when a pool does not store any rbd images. Previously it would return an ENOENT error.

  • Ceph will now issue a health warning if the ‘mon osd down out interval’ config option is set to zero. This warning can be disabled by adding ‘mon warn on osd down out interval zero = false’ to ceph.conf.

值得注意的变化

  • all: improve keepalive detection of failed monitor connections (#7888, Sage Weil)

  • ceph-fuse, libcephfs: pin inodes during readahead, fixing rare crash (#7867, Sage Weil)

  • librbd: make cache writeback a bit less aggressive (Sage Weil)

  • librbd: make symlink for qemu to detect librbd in RPM (#7293, Josh Durgin)

  • mon: allow ‘hashpspool’ pool flag to be set and unset (Loic Dachary)

  • mon: commit paxos state only after entire quorum acks, fixing rare race where prior round state is readable (#7736, Sage Weil)

  • mon: make elections and timeouts a bit more robust (#7212, Sage Weil)

  • mon: prevent extreme pool split operations (Greg Farnum)

  • mon: wait for quorum for get_version requests to close rare pool creation race (#7997, Sage Weil)

  • mon: warn on ‘mon osd down out interval = 0’ (#7784, Joao Luis)

  • msgr: fix byte-order for auth challenge, fixing auth errors on big-endian clients (#7977, Dan Mick)

  • msgr: fix occasional crash in authentication code (usually triggered by librbd) (#6840, Josh Durgin)

  • msgr: fix rebind() race (#6992, Xihui He)

  • osd: avoid timeouts during slow PG deletion (#6528, Samuel Just)

  • osd: fix bug in pool listing during recovery (#6633, Samuel Just)

  • osd: fix queue limits, fixing recovery stalls (#7706, Samuel Just)

  • osd: fix rare peering crashes (#6722, #6910, Samuel Just)

  • osd: fix rare recovery hang (#6681, Samuel Just)

  • osd: improve error handling on journal errors (#7738, Sage Weil)

  • osd: reduce load on the monitor from OSDMap subscriptions (Greg Farnum)

  • osd: rery GetLog on peer osd startup, fixing some rare peering stalls (#6909, Samuel Just)

  • osd: reset journal state on remount to fix occasional crash on OSD startup (#8019, Sage Weil)

  • osd: share maps with peers more aggressively (Greg Farnum)

  • rbd: make it harder to delete an rbd image that is currently in use (#7076, Ilya Drymov)

  • rgw: deny writes to secondary zone by non-system users (#6678, Yehuda Sadeh)

  • rgw: do’nt log system requests in usage log (#6889, Yehuda Sadeh)

  • rgw: fix bucket recreation (#6951, Yehuda Sadeh)

  • rgw: fix Swift range response (#7099, Julien Calvet, Yehuda Sadeh)

  • rgw: fix URL escaping (#8202, Yehuda Sadeh)

  • rgw: fix whitespace trimming in http headers (#7543, Yehuda Sadeh)

  • rgw: make multi-object deletion idempotent (#7346, Yehuda Sadeh)

更多详细信息，请参阅the complete changelog.

v0.67.7 “Dumpling”

This Dumpling point release fixes a few critical issues in v0.67.6.

All v0.67.6 users are urgently encouraged to upgrade. We also recommend that all v0.67.5 (or older) users upgrade.

升级

  • Once you have upgraded a radosgw instance or OSD to v0.67.7, you should not downgrade to a previous version.

值得注意的变化

  • ceph-disk: additional unit tests

  • librbd: revert caching behavior change in v0.67.6

  • osd: fix problem reading xattrs due to incomplete backport in v0.67.6

  • radosgw-admin: fix reading object policy

更多详细信息，请参阅the complete changelog.

v0.67.6 “Dumpling”

This Dumpling point release contains a number of important fixed for the OSD, monitor, and radosgw. Most significantly, a change that forces large object attributes to spill over into leveldb has been backported that can prevent objects and the cluster from being damaged by large attributes (which can be induced via the radosgw). There is also a set of fixes that improves data safety and RADOS semantics when the cluster becomes full and then non-full.

We recommend that all 0.67.x Dumpling users skip this release and upgrade to v0.67.7.

升级

  • The OSD has long contained a feature that allows large xattrs to spill over into the leveldb backing store in situations where not all local file systems are able to store them reliably. This option is now enabled unconditionally in order to avoid rare cases where storing large xattrs renders the object unreadable. This is known to be triggered by very large multipart objects, but could be caused by other workloads as well. Although there is some small risk that performance for certain workloads will degrade, it is more important that data be retrievable. Note that newer versions of Ceph (e.g., firefly) do some additional work to avoid the potential performance regression in this case, but that is current considered too complex for backport to the Dumpling stable series.

  • It is very dangerous to downgrade from v0.67.6 to a prior version of Dumpling. If the old version does not have ‘filestore xattr use omap = true’ it may not be able to read all xattrs for an object and can cause undefined behavior.

Notable changes

  • ceph-disk: misc bug fixes, particularly on RHEL (Loic Dachary, Alfredo Deza, various)

  • ceph-fuse, libcephfs: fix crash from read over certain sparseness patterns (Sage Weil)

  • ceph-fuse, libcephfs: fix integer overflow for sync reads racing with appends (Sage Weil)

  • ceph.spec: fix udev rule when building RPM under RHEL (Derek Yarnell)

  • common: fix crash from bad format from admin socket (Loic Dachary)

  • librados: add optional timeouts (Josh Durgin)

  • librados: do not leak budget when resending localized or redirected ops (Josh Durgin)

  • librados, osd: fix and improve full cluster handling (Josh Durgin)

  • librbd: fix use-after-free when updating perfcounters during image close (Josh Durgin)

  • librbd: remove limit on objects in cache (Josh Durgin)

  • mon: avoid on-disk full OSDMap corruption from pg_temp removal (Sage Weil)

  • mon: avoid stray pg_temp entries from pool deletion race (Joao Eduardo Luis)

  • mon: do not generate spurious MDSMaps from laggy daemons (Joao Eduardo Luis)

  • mon: fix error code from ‘osd rm|down|out|in …’ commands (Loic Dachary)

  • mon: include all health items in summary output (John Spray)

  • osd: fix occasional race/crash during startup (Sage Weil)

  • osd: ignore stray OSDMap messages during init (Sage Weil)

  • osd: unconditionally let xattrs overflow into leveldb (David Zafman)

  • rados: fix a few error checks for the CLI (Josh Durgin)

  • rgw: convert legacy bucket info objects on demand (Yehuda Sadeh)

  • rgw: fix bug causing system users to lose privileges (Yehuda Sadeh)

  • rgw: fix CORS bugs related to headers and case sensitivity (Robin H. Johnson)

  • rgw: fix multipart object listing (Yehuda Sadeh)

  • rgw: fix racing object creations (Yehuda Sadeh)

  • rgw: fix racing object put and delete (Yehuda Sadeh)

  • rgw: fix S3 auth when using response-* query string params (Sylvain Munaut)

  • rgw: use correct secret key for POST authentication (Robin H. Johnson)

更多详细信息，请参阅the complete changelog.

v0.67.5 “Dumpling”

This release includes a few critical bug fixes for the radosgw, including a fix for hanging operations on large objects. There are also several bug fixes for radosgw multi-site replications, and a few backported features. Also, notably, the ‘osd perf’ command (which dumps recent performance information about active OSDs) has been backported.

We recommend that all 0.67.x Dumpling users upgrade.

Notable changes

  • ceph-fuse: fix crash in caching code

  • mds: fix looping in populate_mydir()

  • mds: fix standby-replay race

  • mon: accept ‘osd pool set …’ as string

  • mon: backport: ‘osd perf’ command to dump recent OSD performance stats

  • osd: add feature compat check for upcoming object sharding

  • osd: fix osd bench block size argument

  • rbd.py: 增加父名称大小限制

  • rgw: 回滚: 允许在支持的 keystone 角色中使用通配符

  • rgw: 回滚: 改善 swift COPY 行为

  • rgw: 回滚: 默认情况下记录并打开管理员套接字

  • rgw: 回滚: 验证 S3 令牌与 keystone

  • rgw: 修复桶删除

  • rgw: 修复分块 PUT 失败的客户端错误代码

  • rgw: 修复大型对象 GET 上的挂起

  • rgw: 修复罕见的 use-after-free

  • rgw: 各种容灾修复

  • sysvinit, upstart: 防止使用两个 init 系统启动守护进程

更多详细信息，请参阅the complete changelog.

v0.67.4 “Dumpling”

这个点发布修复了 radosgw、keystone 身份验证令牌缓存和 CORS 的重要性能问题。所有用户（尤其是 rgw 用户）都被鼓励升级。

Notable changes

  • crush: 修复缓存的名称无效化

  • crushtool: 在非唯一桶 ID 上不会崩溃

  • mds: 解码 LogEvents 时更加小心

  • mds: 修复堆检查调试命令

  • mon: 避免重建旧的全 osdmaps

  • mon: 修复 “ceph crush move …”

  • mon: 修复 “ceph osd crush reweight …”

  • mon: 在修剪期间修复全 osdmaps 的写入

  • mon: 限制事务大小

  • mon: 防止同时未管理的和池快照

  • osd: 禁用 xattr 大小限制（防止上传大型 rgw 对象）

  • osd: 修复恢复操作限制

  • osd: 修复对非常慢的请求的日志消息限制

  • rgw: 在完成写入之前清空挂起的请求

  • rgw: 修复 CORS

  • rgw: 修复低效的 list::size() 使用

  • rgw: 修复 keystone 令牌过期

  • rgw: 修复轻微的内存泄漏

  • rgw: 修复缓冲区的空终止

更多详细信息，请参阅the complete changelog.

v0.67.3 “Dumpling”

这个点发布修复了 OSD 的几个重要性能回归（CPU 和磁盘利用率），以及几个其他重要但不太常见的问题。我们建议所有生产用户升级。

值得注意的变化

  • ceph-disk: 创建后 partprobe 日志分区

  • ceph-disk: 挂载时指定文件系统类型

  • ceph-post-file: 新实用程序，帮助与 ceph 开发者共享日志和其他文件

  • libcephfs: 修复截断与预读竞争条件（崩溃）

  • mds: 修复 flock/fcntl 锁死

  • mds: 遇到预-Dumpling 回指时修复重新加入循环

  • mon: 在选举阶段允许名称和地址发现

  • mon: 在 Paxos store_state 后始终刷新（修复恢复边缘情况）

  • mon: 修复 osd 字节计数的 off-by-4x 错误

  • osd: 默认情况下禁用 ‘pg log keys debug’ (#9514 Greg Farnum)

  • osd: 添加选项以禁用限制

  • osd: 避免使用 leveldb 迭代器进行 pg 日志追加和修剪

  • osd: 修复 readdir_r 调用

  • osd: 使用 fdatasync 而不是 sync

  • radosgw: 修复 sysvinit 脚本的返回状态

  • rbd: 重许可为 LGPL2

  • rgw: 在多部分上传中刷新挂起的数据

  • rgw: 在 S3 POST 期间重新检查对象名称

  • rgw: 重新排序初始化/启动

  • rpm: 修复调试信息软件包构建

更多详细信息，请参阅the complete changelog.

v0.67.2 “Dumpling”

这是一个 Dumpling 的重要点发布。最值得注意的是，它修复了从 v0.56.x Bobtail 直接升级到 v0.67.x Dumpling（中间不停止在 v0.61.x Cuttlefish）时的问题。它还修复了 CLI 解析 CEPH_ARGS 环境变量的问题、 ceph-osd 守护进程的高 CPU 利用率，并清理了 radosgw 关闭序列。

值得注意的变化

  • objecter: 当集群从全状态变为非全状态时重新发送挂起请求

  • ceph: 解析 CEPH_ARGS 环境变量

  • librados: 修复小内存泄漏

  • osd: 在升级时删除旧日志对象（修复 bobtail -> dumpling 跳转）

  • osd: 通过配置选项禁用 PGLog::check()（修复 CPU 烧毁）

  • rgw: 在关闭时清空请求

  • rgw: 关闭时的一些内存泄漏

更多详细信息，请参阅the complete changelog.

v0.67.1 “Dumpling”

这是一个 Dumpling 的次要点发布，它修复了 OpenStack 和 librbd 挂起在禁用缓存时的问题。

Notable changes

  • librados, librbd: 修复 python 绑定构造函数，用于某些用法（特别是 OpenStack 使用的用法）

  • librados, librbd: 修复禁用缓存时 aio_flush 唤醒

  • librados: 修复 aio 完成引用计数的锁定

  • 修复 ‘ceph --admin-daemon …’ 命令的错误代码

  • 修复 ‘ceph daemon … config set …’ 命令，用于布尔配置选项

更多详细信息，请参阅the complete changelog.

v0.67 “Dumpling”

这是 Ceph 的第四个主要版本，代号为“Dumpling”。此版本的头条功能包括：

  • radosgw 的多站点支持。这包括能够在同一或不同的 Ceph 集群中设置单独的“区域”，这些区域共享单个 S3/Swift 桶/容器命名空间。

  • 用于 Ceph 集群管理的 RESTful API 端点。

  • librados 中的对象命名空间。

Upgrade Sequencing

可以从 Cuttlefish 到 Dumpling 进行滚动升级。

  1. 升级所有将使用命令行 ‘ceph’ 实用程序的节点上的 ceph-common。

  2. 升级所有监视器（升级 ceph 软件包，重新启动 ceph-mon 守护进程）。这可以一次一个守护进程或主机发生。

  3. 升级所有 osds（升级 ceph 软件包，重新启动 ceph-osd 守护进程）。这可以一次一个守护进程或主机发生。

  4. 升级 radosgw（升级 radosgw 软件包，重新启动 radosgw 守护进程）。

从 v0.66 升级

  • 存在监视器内部协议更改，这意味着 v0.67 ceph-mon 守护进程无法与 v0.66 或更早版本的守护进程通信。我们建议同时升级所有监视器（或相对快速地连续升级），以最大程度地减少停机时间。

  • ‘ceph status --format=json’ 或 ‘ceph -s --format=json’ 的输出已更改为以更结构化和更易于使用的格式返回状态信息。

  • ‘ceph pg dump_stuck [threshold]’ 命令现在不需要阈值参数的前缀，即 --threshold 或 -t。

  • 许多 ceph 命令现在输出格式化信息；使用 ‘--format=<format>’，其中 <format> 可以是 ‘json’、‘json-pretty’、‘xml’ 或 ‘xml-pretty’。

  • ‘ceph pg <pgid> …’ 命令（例如 ‘ceph pg <pgid> query’）已弃用，改为使用 ‘ceph tell <pgid> …’。这使得 ‘ceph pg <command> <pgid>’ 和 ‘ceph pg <pgid> <command>’ 之间的区别不那么尴尬，因为它更清楚地表明 ‘tell’ 命令正在与提供放置组的 OSD 通信，而不是监视器。

  • ‘ceph --admin-daemon <path> <command …>’ 现在接受命令和参数作为单个字符串或单独的参数。它现在只接受跨多个参数的命令。这意味着任何执行以下操作：

    ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok 'config set debug_ms 1'

    需要删除引号。另外，请注意，上述内容现在可以缩短为：

    ceph daemon osd.0 config set debug_ms 1

  • radosgw 的令牌以前不一致地记录为 ‘mon = allow r’ 或 ‘mon = allow rw’。‘mon = allow rw’ 是 radosgw 创建自己的池所必需的。所有文档都已相应更新。

  • radosgw 复制对象操作可能在操作期间返回额外的进度信息。此时它只会在执行跨区域复制操作时发生。S3 响应现在将返回 <CopyResult> 容器下的额外 <Progress> 字段。Swift 响应现在将进度作为 json 数组发送。

  • 在 v0.66 和 v0.65 中，新的池默认启用 HASHPSPOOL 池标志，以更好地在 OSD 之间分配。此功能的支持是在 v0.59 中引入的，并且 Linux 内核版本 v3.9 也引入了此功能。如果您希望从较旧的内核访问集群，请在创建集群或创建新池之前在 ceph.conf 中设置 ‘osd pool default flag hashpspool = false’ 选项。请注意，集群中任何启用此标志的池的存在都将使 OSD 需要所有客户端的支持。

  • ceph-osd now requires a max file descriptor limit (e.g., ulimit -n ...) of at least filestore_wbthrottle_(xfs|btrfs)_inodes_hard_limit (5000 by default) in order to accommodate the new write back throttle system. On Ubuntu, upstart now sets the fd limit to 32k. On other platforms, the sysvinit script will set it to 32k by default (still overridable via max_open_files). If this field has been customized in ceph.conf it should likely be adjusted upwards.

Upgrading from v0.61 “Cuttlefish”

In addition to the above notes about upgrading from v0.66:

  • There has been a huge revamp of the ‘ceph’ command-line interface implementation. The ceph-common client library needs to be upgrade before ceph-mon is restarted in order to avoid problems using the CLI (the old ceph client utility cannot talk to the new ceph-mon).

  • The CLI is now very careful about sending the ‘status’ one-liner output to stderr and command output to stdout. Scripts relying on output should take care.

  • The ‘ceph osd tell …’ and ‘ceph mon tell …’ commands are no longer supported. Any callers should use:

    ceph tell osd.<id or *> ...
ceph tell mon.<id or name or *> ...

    The ‘ceph mds tell …’ command is still there, but will soon also transition to ‘ceph tell mds.<id or name or *> …’

  • The ‘ceph osd crush add …’ command used to take one of two forms:

    ceph osd crush add 123 osd.123 <weight> <location ...>
ceph osd crush add osd.123 <weight> <location ...>

    This is because the id and crush name are redundant. Now only the simple form is supported, where the osd name/id can either be a bare id (integer) or name (osd.<id>):

    ceph osd crush add osd.123 <weight> <location ...>
ceph osd crush add 123 <weight> <location ...>

  • There is now a maximum RADOS object size, configurable via ‘osd max object size’, defaulting to 100 GB. Note that this has no effect on RBD, CephFS, or radosgw, which all stripe over objects. If you are using librados and storing objects larger than that, you will need to adjust ‘osd max object size’, and should consider using smaller objects instead.

  • ‘osd min down {reporters|reports}’ 配置选项已被重命名为 ‘mon osd min down {reporters|reports}’，并且文档已更新以反映这些选项适用于监视器（处理故障报告）而不是 OSD。如果您调整了这些设置，请相应地更新ceph.conf。

Notable changes since v0.66

  • mon: sync improvements (performance and robustness)

  • mon: many bug fixes (paxos and services)

  • mon: fixed bugs in recovery and io rate reporting (negative/large values)

  • mon: collect metadata on osd performance

  • mon: generate health warnings from slow or stuck requests

  • mon: expanded --format=<json|xml|…> support for monitor commands

  • mon: scrub function for verifying data integrity

  • mon, osd: fix old osdmap trimming logic

  • mon: enable leveldb caching by default

  • mon: more efficient storage of PG metadata

  • ceph-rest-api: RESTful endpoint for administer cluster (mirrors CLI)

  • rgw: multi-region support

  • rgw: infrastructure to support georeplication of bucket and user metadata

  • rgw: infrastructure to support georeplication of bucket data

  • rgw: COPY object support between regions

  • rbd: /etc/ceph/rbdmap file for mapping rbd images on startup

  • osd: many bug fixes

  • osd: limit number of incremental osdmaps sent to peers (could cause osds to be wrongly marked down)

  • osd: more efficient small object recovery

  • osd, librados: support for object namespaces

  • osd: automatically enable xattrs on leveldb as necessary

  • mds: fix bug in LOOKUPINO (used by nfs reexport)

  • mds: fix O_TRUNC locking

  • msgr: fixed race condition in inter-osd network communication

  • msgr: fixed various memory leaks related to network sessions

  • ceph-disk: fixes for unusual device names, partition detection

  • hypertable: fixes for hypertable CephBroker bindings

  • use SSE4.2 crc32c instruction if present

Notable changes since v0.61 “Cuttlefish”

  • 添加 ‘config get’ 管理套接字命令

  • ceph-conf: --show-config-value now reflects daemon defaults

  • ceph-disk: 添加 ‘[un]suppress-active DEV’ 命令

  • ceph-disk: avoid mounting over an existing osd in /var/lib/ceph/osd/*

  • ceph-disk: fixes for unusual device names, partition detection

  • ceph-disk: improved handling of odd device names

  • ceph-disk: many fixes for RHEL/CentOS, Fedora, wheezy

  • ceph-disk: simpler, more robust locking

  • ceph-fuse, libcephfs: fix a few caps revocation bugs

  • ceph-fuse, libcephfs: fix read zeroing at EOF

  • ceph-fuse, libcephfs: fix request refcounting bug (hang on shutdown)

  • ceph-fuse, libcephfs: fix truncatation bug on >4MB files (Yan, Zheng)

  • ceph-fuse, libcephfs: fix for cap release/hang

  • ceph-fuse: 添加 ioctl 支持

  • ceph-fuse: fixed long-standing O_NOATIME vs O_LAZY bug

  • ceph-rest-api: RESTful endpoint for administer cluster (mirrors CLI)

  • ceph, librados: fix resending of commands on mon reconnect

  • 守护进程: 根据需要创建 /var/run/ceph

  • debian wheezy: 修复 udev 规则

  • debian, specfile: 打包清理

  • debian: 修复升级时 upstart 行为

  • debian: rgw: 卸载时停止守护进程

  • debian: 卸载时停止守护进程；修复依赖关系

  • hypertable: fixes for hypertable CephBroker bindings

  • librados python 绑定清理

  • librados python: fix xattrs > 4KB (Josh Durgin)

  • librados: configurable max object size (default 100 GB)

  • librados: new calls to administer the cluster

  • librbd: 能够从本地副本读取

  • librbd: locking tests (Josh Durgin)

  • librbd: 为新创建的图像设置默认选项/功能（例如，通过 qemu-img）

  • librbd: 并行化删除、回滚、扁平化、复制、调整大小

  • 从静态代码分析中修复了许多许多的修复 (Danny Al-Gaaf)

  • mds: fix O_TRUNC locking

  • mds: fix bug in LOOKUPINO (used by nfs reexport)

  • mds: 修复客户端重新启动后的罕见挂起

  • mds: fix several bugs (Yan, Zheng)

  • mds: many backpointer improvements (Yan, Zheng)

  • mds: 许多修复 mds 集群问题

  • mds: misc stability fixes (Yan, Zheng, Greg Farnum)

  • mds: new robust open-by-ino support (Yan, Zheng)

  • mds: support robust lookup by ino number (good for NFS) (Yan, Zheng)

  • mon, ceph: huge revamp of CLI and internal admin API. (Dan Mick)

  • mon, osd: fix old osdmap trimming logic

  • mon, osd: many memory leaks fixed

  • mon: 更好的修剪/压缩行为

  • mon: collect metadata on osd performance

  • mon: enable leveldb caching by default

  • mon: expanded --format=<json|xml|…> support for monitor commands

  • mon: 修复选举超时

  • mon: 修复 leveldb 压缩、修剪

  • mon: 修复启动 fork 行为

  • mon: 修复 ‘ceph df’ 输出中的单位

  • mon: 修复从 CLI 命令验证 mds id

  • mon: fixed bugs in recovery and io rate reporting (negative/large values)

  • mon: generate health warnings from slow or stuck requests

  • mon: many bug fixes (paxos and services, sync)

  • mon: many stability fixes (Joao Luis)

  • mon: more efficient storage of PG metadata

  • mon: 新的 --extract-monmap 以便灾难恢复

  • mon: new capability syntax

  • mon: scrub function for verifying data integrity

  • mon: simplify PaxosService vs Paxos interaction, fix readable/writeable checks

  • mon: sync improvements (performance and robustness)

  • mon: tuning, performance improvements

  • msgr: fix various memory leaks

  • msgr: fixed race condition in inter-osd network communication

  • msgr: fixed various memory leaks related to network sessions

  • osd, librados: support for object namespaces

  • osd, mon: 可选地将 leveldb 事务转储到日志

  • osd: automatically enable xattrs on leveldb as necessary

  • osd: avoid osd flapping from asymmetric network failure

  • osd: break blacklisted client watches (David Zafman)

  • osd: 关闭窄 journal 竞争

  • osd: do not use fadvise(DONTNEED) on XFS (data corruption on power cycle)

  • osd: 修复操作排序错误

  • osd: 修复从 bobtail 升级后的拆分处理

  • osd: 修复 osds 的错误标记

  • osd: 修复在清理非常大的对象时内部心跳超时

  • osd: 修复内存/网络效率低下（在深度清理期间）

  • osd: fixed problem with front-side heartbeats and mixed clusters (David Zafman)

  • osd: limit number of incremental osdmaps sent to peers (could cause osds to be wrongly marked down)

  • osd: many bug fixes

  • osd: 监控前和后接口

  • osd: more efficient small object recovery

  • osd: new writeback throttling (for less bursty write performance) (Sam Just)

  • osd: pg log (re)writes are now vastly more efficient (faster peering) (Sam Just)

  • osd: ping/heartbeat on public and private interfaces

  • osd: 优先恢复降级的 PGs

  • osd: re-use partially deleted PG contents when present (Sam Just)

  • osd: recovery and peering performance improvements

  • osd: 恢复部分删除的 PGs

  • osd: 在重新加入集群之前验证前和后网络是否正常工作

  • rados: clonedata 命令用于 cli

  • radosgw-admin: 默认情况下为新用户创建密钥

  • rbd: /etc/ceph/rbdmap file for mapping rbd images on startup

  • rgw: COPY object support between regions

  • rgw: 修复 CORS 错误

  • rgw: 修复锁定问题，用户操作掩码，

  • rgw: fix radosgw-admin buckets list (Yehuda Sadeh)

  • rgw: 修复使用大型、未修剪日志的用法日志扫描

  • rgw: 处理深度 uri 资源

  • rgw: infrastructure to support georeplication of bucket and user metadata

  • rgw: infrastructure to support georeplication of bucket data

  • rgw: multi-region support

  • sysvinit: fix enumeration of local daemons

  • sysvinit: fix osd crush weight calculation when using -a

  • sysvinit: handle symlinks in /var/lib/ceph/osd/*

  • use SSE4.2 crc32c instruction if present

v0.66

升级

  • There is now a configurable maximum rados object size, defaulting to 100 GB. If you are using librados and storing objects larger than that, you will need to adjust ‘osd max object size’, and should consider using smaller objects instead.

Notable changes

  • osd: pg log (re)writes are now vastly more efficient (faster peering) (Sam Just)

  • osd: fixed problem with front-side heartbeats and mixed clusters (David Zafman)

  • mon: tuning, performance improvements

  • mon: simplify PaxosService vs Paxos interaction, fix readable/writeable checks

  • rgw: fix radosgw-admin buckets list (Yehuda Sadeh)

  • mds: support robust lookup by ino number (good for NFS) (Yan, Zheng)

  • mds: fix several bugs (Yan, Zheng)

  • ceph-fuse, libcephfs: fix truncatation bug on >4MB files (Yan, Zheng)

  • ceph/librados: fix resending of commands on mon reconnect

  • librados python: fix xattrs > 4KB (Josh Durgin)

  • librados: configurable max object size (default 100 GB)

  • msgr: fix various memory leaks

  • ceph-fuse: fixed long-standing O_NOATIME vs O_LAZY bug

  • ceph-fuse, libcephfs: fix request refcounting bug (hang on shutdown)

  • ceph-fuse, libcephfs: fix read zeroing at EOF

  • ceph-conf: --show-config-value now reflects daemon defaults

  • ceph-disk: simpler, more robust locking

  • ceph-disk: avoid mounting over an existing osd in /var/lib/ceph/osd/*

  • sysvinit: handle symlinks in /var/lib/ceph/osd/*

v0.65

升级

  • Huge revamp of the ‘ceph’ command-line interface implementation. The ceph-common client library needs to be upgrade before ceph-mon is restarted in order to avoid problems using the CLI (the old ceph client utility cannot talk to the new ceph-mon).

  • The CLI is now very careful about sending the ‘status’ one-liner output to stderr and command output to stdout. Scripts relying on output should take care.

  • The ‘ceph osd tell …’ and ‘ceph mon tell …’ commands are no longer supported. Any callers should use:

    ceph tell osd.<id or *> ...
ceph tell mon.<id or name or *> ...

    The ‘ceph mds tell …’ command is still there, but will soon also transition to ‘ceph tell mds.<id or name or *> …’

  • The ‘ceph osd crush add …’ command used to take one of two forms:

    ceph osd crush add 123 osd.123 <weight> <location ...>
ceph osd crush add osd.123 <weight> <location ...>

    This is because the id and crush name are redundant. Now only the simple form is supported, where the osd name/id can either be a bare id (integer) or name (osd.<id>):

    ceph osd crush add osd.123 <weight> <location ...>
ceph osd crush add 123 <weight> <location ...>

  • There is now a maximum RADOS object size, configurable via ‘osd max object size’, defaulting to 100 GB. Note that this has no effect on RBD, CephFS, or radosgw, which all stripe over objects.

Notable changes

  • mon, ceph: huge revamp of CLI and internal admin API. (Dan Mick)

  • mon: new capability syntax

  • osd: do not use fadvise(DONTNEED) on XFS (data corruption on power cycle)

  • osd: recovery and peering performance improvements

  • osd: new writeback throttling (for less bursty write performance) (Sam Just)

  • osd: ping/heartbeat on public and private interfaces

  • osd: avoid osd flapping from asymmetric network failure

  • osd: re-use partially deleted PG contents when present (Sam Just)

  • osd: break blacklisted client watches (David Zafman)

  • mon: many stability fixes (Joao Luis)

  • mon, osd: many memory leaks fixed

  • mds: misc stability fixes (Yan, Zheng, Greg Farnum)

  • mds: many backpointer improvements (Yan, Zheng)

  • mds: new robust open-by-ino support (Yan, Zheng)

  • ceph-fuse, libcephfs: fix a few caps revocation bugs

  • librados: new calls to administer the cluster

  • librbd: locking tests (Josh Durgin)

  • ceph-disk: improved handling of odd device names

  • ceph-disk: many fixes for RHEL/CentOS, Fedora, wheezy

  • 从静态代码分析中修复了许多许多的修复 (Danny Al-Gaaf)

  • 守护进程: 根据需要创建 /var/run/ceph

v0.64

升级

  • 新的池现在默认设置 HASHPSPOOL 标志，以更好地在 OSD 之间分配。此功能的支持是在 v0.59 中引入的，并且 Linux 内核版本 v3.9 也引入了此功能。如果您希望从较旧的内核访问集群，请在创建集群或创建新池之前在 ceph.conf 中设置 ‘osd pool default flag hashpspool = false’ 选项。请注意，集群中任何启用此标志的池的存在都将使 OSD 需要所有客户端的支持。

Notable changes

  • osd: 监控前和后接口

  • osd: 在重新加入集群之前验证前和后网络是否正常工作

  • osd: 修复内存/网络效率低下（在深度清理期间）

  • osd: 修复 osds 的错误标记

  • mon: 修复启动 fork 行为

  • mon: 修复选举超时

  • mon: 更好的修剪/压缩行为

  • mon: 修复 ‘ceph df’ 输出中的单位

  • mon, osd: 一些内存泄漏

  • librbd: 为新创建的图像设置默认选项/功能（例如，通过 qemu-img）

  • mds: 许多修复 mds 集群问题

  • mds: 修复客户端重新启动后的罕见挂起

  • ceph-fuse: 添加 ioctl 支持

  • ceph-fuse/libcephfs: 修复 cap 释放/挂起问题

  • rgw: 处理深度 uri 资源

  • rgw: 修复 CORS 错误

  • ceph-disk: 添加 ‘[un]suppress-active DEV’ 命令

  • debian: rgw: 卸载时停止守护进程

  • debian: 修复升级时 upstart 行为

v0.63

升级

  • ‘osd min down {reporters|reports}’ 配置选项已被重命名为 ‘mon osd min down {reporters|reports}’，并且文档已更新以反映这些选项适用于监视器（处理故障报告）而不是 OSD。如果您调整了这些设置，请相应地更新ceph.conf。

值得注意的变化

  • librbd: 并行化删除、回滚、扁平化、复制、调整大小

  • librbd: 能够从本地副本读取

  • osd: 恢复部分删除的 PGs

  • osd: 优先恢复降级的 PGs

  • osd: 修复在清理非常大的对象时内部心跳超时

  • osd: 关闭窄 journal 竞争

  • rgw: 修复使用大型、未修剪日志的用法日志扫描

  • rgw: 修复锁定问题，用户操作掩码，

  • initscript: 使用 -a 时修复 osd crush weight 计算

  • initscript: 修复本地守护进程的枚举

  • mon: 对 paxos、同步进行了一些修复

  • mon: 新的 --extract-monmap 以便灾难恢复

  • mon: 修复 leveldb 压缩、修剪

  • 添加 ‘config get’ 管理套接字命令

  • rados: clonedata 命令用于 cli

  • debian: 卸载时停止守护进程；修复依赖关系

  • debian wheezy: 修复 udev 规则

  • many many small fixes from coverity scan

v0.62

值得注意的变化

  • mon: 修复从 CLI 命令验证 mds id

  • osd: 修复操作排序错误

  • osd, mon: 可选地将 leveldb 事务转储到日志

  • osd: 修复从 bobtail 升级后的拆分处理

  • debian, specfile: 打包清理

  • radosgw-admin: 默认情况下为新用户创建密钥

  • librados python 绑定清理

  • 其他代码清理

由 Ceph 基金会带给您

Ceph 文档是一个社区资源，由非盈利的 Ceph 基金会资助和托管Ceph Foundation. 如果您想支持这一点和我们的其他工作，请考虑加入现在加入.