CephFS supports asynchronous snapshot mirroring [1][2] since RHCS 5 (Ceph Pacific). This feature is enabled by the use of the `cephfs-mirror` daemon, and one daemon can be pre-configured per Ceph cluster. The daemon can be configured to replicate between filesystems on the same cluster, or to a peered Ceph cluster. Enabling this would help OpenStack Manila customers benefit from disaster recovery protections. 

The primary objective of this Epic is to ensure that Manila's CephFS interactions are unaffected when mirroring is enabled; and we also need to identify what needs to be done to recover Manila shares after a mirror has been "promoted".

As to whether mirroring can be enabled/orchestrated with Manila itself:

OpenStack Manila supports share replication [3] with a flexible definition for asynchronous data mirroring of two types: "dr" for disaster recovery where the destination is unavailable to mount, but is continuously updated; and "readable" where the destination can be mounted. The RTO/RPO of the replication isn't controlled by Manila intentionally for flexibility. There's however, an expectation from share backends that:

• They support setting up a per-share replica
• They support replicating snapshots
• They support "promoting" a replica which reverses the direction of replication

While the third option is not possible with CephFS replication today, there's may be value in providing a mechanism to setup per-share replication, and allow creation of snapshots that are then automatically replicated. 

An alternative is to build a CephFS mirroring strategy that's totally transparent to OpenStack Manila; i.e., the base CephFS filesystem can be configured to be mirrored, which encompasses all subvolumes. This can be done today outside of Manila; but there's no visibility whatsoever to Manila consumers. We believe the Manila CephFS driver should at least be able to detect the presence of mirroring and tag the storage pool/filesystem as being replicated - which allows OpenStack administrators to direct provisioning to this on-demand (via a driver-specific, scoped share type extra spec). 

[1] https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/5/html-single/file_system_guide/index#ceph-file-system-mirrors

[2] https://docs.ceph.com/en/latest/dev/cephfs-mirroring/ 

[3] https://docs.openstack.org/manila/latest/admin/shared-file-systems-share-replication.html 

What are the use cases this RFE is solving?
High Level view on how the feature works
Is this feature driver dependent or driver related?
Are there any known limitations? (e.g multi attach + encryption)
Is a CLI change required, does the openstack cmd support it?
Does this RFE impact / need to be included into the control plane podification?
Does this RFE benefit/impact DCN?
Does this RFE benefit/impact shift on stack?
Can this feature be turned on or used in an existing environment?
Does this feature affect another DFG or product?
Does this feature depend on another RFE?
How will the feature affect Upgrades?
How will the feature affect performance or scaling?
What are the test cases for this RFE?
Are there CI implications?
Does it have documentation impact and require early planning with the doc team?
Are there known packaging challenges?
Are there any security considerations?
How much upstream resistance might there be to this feature?
Will this feature require new or different support skills?
Will this be required for knowledge transfer to GSS?
Will this feature impact existing partners or certification programs?
API Deprecation/Compatibility?
Are any GUI impact/changes required (Horizon)?