数据库 MongoDB

MongoDB误操作后使用oplog恢复数据

Posted in MongoDB onApril 11, 2022

数据全备

mongodump -h 172.16.254.133 --port 27017 -o /mongodb/backup/backup

模拟故障

写入数据

handong1:PRIMARY> for (var i = 1; i <= 100; i++) {
...    db.test.insert( { id : i , name: "handong" , name1:"handong", name2:"handong", name3:"handong"} )
... }
WriteResult({ "nInserted" : 1 })
handong1:PRIMARY> 
handong1:PRIMARY> db.test.count()
100

模拟误操作

handong1:PRIMARY> db.test.remove({})
WriteResult({ "nRemoved" : 100 })
handong1:PRIMARY> db.test.count()
0

所有文档被误删除。

恢复步骤

备份oplog

mongodump -h 172.16.254.133 --port 27017 -d local -c oplog.rs -o /mongodb/backup
2021-04-30T18:32:29.077+0800	writing local.oplog.rs to /mongodb/backup/local/oplog.rs.bson
2021-04-30T18:32:32.039+0800	local.oplog.rs  7108
2021-04-30T18:32:35.038+0800	local.oplog.rs  17912
2021-04-30T18:32:38.041+0800	local.oplog.rs  28226
2021-04-30T18:32:41.039+0800	local.oplog.rs  38642
2021-04-30T18:32:44.042+0800	local.oplog.rs  50679
2021-04-30T18:32:47.040+0800	local.oplog.rs  64001
2021-04-30T18:32:50.040+0800	local.oplog.rs  77265
2021-04-30T18:32:53.038+0800	local.oplog.rs  89739
2021-04-30T18:32:56.038+0800	local.oplog.rs  102449
2021-04-30T18:32:57.697+0800	local.oplog.rs  132459
2021-04-30T18:32:57.697+0800	done dumping local.oplog.rs (132459 documents)

解析oplog

bsondump /mongodb/backup/local/oplog.rs.bson  > /mongodb/backup/local/local.log
2021-04-30T18:34:27.612+0800	132459 objects found

将oplog备份和全备复制到standalone机

scp -r backup/ 172.16.254.130:/mongodb/backup/

scp -r local/ 172.16.254.130:/mongodb/backup/backup

查找误操作时间点

通过查看解析完的日志local.log发现误操作的时间点在1619778429。

进行数据恢复

mongorestore -h 172.16.254.130 --port 27017 --oplogReplay --oplogLimit 1619778429:1 /mongodb/backup/backup
2021-04-30T19:00:11.099+0800	preparing collections to restore from
2021-04-30T19:00:11.100+0800	don't know what to do with file "/mongodb/backup/backup/local/111.log", skipping...
2021-04-30T19:00:11.100+0800	don't know what to do with file "/mongodb/backup/backup/local/local.log", skipping...
2021-04-30T19:00:11.116+0800	reading metadata for db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.metadata.json
2021-04-30T19:00:11.117+0800	reading metadata for ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.metadata.json
2021-04-30T19:00:11.119+0800	reading metadata for db3.db3 from /mongodb/backup/backup/db3/db3.metadata.json
2021-04-30T19:00:11.119+0800	reading metadata for ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.metadata.json
2021-04-30T19:00:11.170+0800	restoring ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.bson
2021-04-30T19:00:11.187+0800	restoring ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.bson
2021-04-30T19:00:11.391+0800	restoring db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.bson
2021-04-30T19:00:11.580+0800	restoring db3.db3 from /mongodb/backup/backup/db3/db3.bson
2021-04-30T19:00:11.661+0800	no indexes to restore
2021-04-30T19:00:11.661+0800	finished restoring db3.db3 (6 documents, 0 failures)
2021-04-30T19:00:11.662+0800	reading metadata for db5.test from /mongodb/backup/backup/db5/test.metadata.json
2021-04-30T19:00:12.545+0800	restoring db5.test from /mongodb/backup/backup/db5/test.bson
2021-04-30T19:00:12.548+0800	no indexes to restore
2021-04-30T19:00:12.548+0800	finished restoring db5.test (0 documents, 0 failures)
2021-04-30T19:00:12.548+0800	reading metadata for db4.db4 from /mongodb/backup/backup/db4/db4.metadata.json
2021-04-30T19:00:13.784+0800	no indexes to restore
2021-04-30T19:00:13.784+0800	finished restoring ycsb1.usertable (30370 documents, 0 failures)
2021-04-30T19:00:13.785+0800	reading metadata for db4.test1 from /mongodb/backup/backup/db4/test1.metadata.json
2021-04-30T19:00:14.099+0800	[###################.....]          ycsb.usertable  47.8MB/58.9MB  (81.2%)
2021-04-30T19:00:14.099+0800	[........................]  db4.rcmd_1_tag_li_liao  79.8MB/3.32GB   (2.3%)
2021-04-30T19:00:14.099+0800	
2021-04-30T19:00:14.843+0800	[########################]  ycsb.usertable  58.9MB/58.9MB  (100.0%)
2021-04-30T19:00:14.843+0800	no indexes to restore
2021-04-30T19:00:14.843+0800	finished restoring ycsb.usertable (43458 documents, 0 failures)
2021-04-30T19:00:15.339+0800	restoring db4.db4 from /mongodb/backup/backup/db4/db4.bson
2021-04-30T19:00:17.077+0800	restoring db4.test1 from /mongodb/backup/backup/db4/test1.bson
2021-04-30T19:00:17.103+0800	[#.......................]  db4.rcmd_1_tag_li_liao  167MB/3.32GB  (4.9%)
2021-04-30T19:00:17.104+0800	[#.......................]                 db4.db4  8.11MB/137MB  (5.9%)
2021-04-30T19:00:17.104+0800	[........................]               db4.test1      0B/104MB  (0.0%)
2021-04-30T19:00:17.104+0800	
2021-04-30T19:00:20.099+0800	[#.......................]  db4.rcmd_1_tag_li_liao  204MB/3.32GB   (6.0%)
2021-04-30T19:00:20.099+0800	[####....................]                 db4.db4  24.5MB/137MB  (18.0%)
2021-04-30T19:00:20.099+0800	[#.......................]               db4.test1  4.47MB/104MB   (4.3%)
2021-04-30T19:00:20.099+0800	
2021-04-30T19:00:23.099+0800	[#.......................]  db4.rcmd_1_tag_li_liao  272MB/3.32GB   (8.0%)
2021-04-30T19:00:23.099+0800	[######..................]                 db4.db4  39.7MB/137MB  (29.1%)
2021-04-30T19:00:23.099+0800	[####....................]               db4.test1  20.1MB/104MB  (19.3%)
2021-04-30T19:00:23.099+0800	
2021-04-30T19:00:26.102+0800	[##......................]  db4.rcmd_1_tag_li_liao  355MB/3.32GB  (10.4%)
2021-04-30T19:00:26.102+0800	[##########..............]                 db4.db4  58.0MB/137MB  (42.5%)
2021-04-30T19:00:26.102+0800	[########................]               db4.test1  38.1MB/104MB  (36.7%)
2021-04-30T19:00:26.102+0800	
2021-04-30T19:00:29.098+0800	[##......................]  db4.rcmd_1_tag_li_liao  401MB/3.32GB  (11.8%)
2021-04-30T19:00:29.098+0800	[############............]                 db4.db4  73.1MB/137MB  (53.5%)
2021-04-30T19:00:29.098+0800	[###########.............]               db4.test1  51.8MB/104MB  (49.8%)
2021-04-30T19:00:29.098+0800	
2021-04-30T19:00:32.097+0800	[###.....................]  db4.rcmd_1_tag_li_liao  494MB/3.32GB  (14.5%)
2021-04-30T19:00:32.097+0800	[###############.........]                 db4.db4  90.8MB/137MB  (66.5%)
2021-04-30T19:00:32.097+0800	[###############.........]               db4.test1  67.3MB/104MB  (64.7%)
2021-04-30T19:00:32.097+0800	
2021-04-30T19:00:35.100+0800	[###.....................]  db4.rcmd_1_tag_li_liao  556MB/3.32GB  (16.3%)
2021-04-30T19:00:35.100+0800	[###################.....]                 db4.db4   110MB/137MB  (80.5%)
2021-04-30T19:00:35.100+0800	[###################.....]               db4.test1  86.1MB/104MB  (82.8%)
2021-04-30T19:00:35.100+0800	
2021-04-30T19:00:38.097+0800	[####....................]  db4.rcmd_1_tag_li_liao  620MB/3.32GB  (18.2%)
2021-04-30T19:00:38.097+0800	[#####################...]                 db4.db4   124MB/137MB  (91.1%)
2021-04-30T19:00:38.097+0800	[#######################.]               db4.test1   101MB/104MB  (96.7%)
2021-04-30T19:00:38.097+0800	
2021-04-30T19:00:39.023+0800	[########################]  db4.test1  104MB/104MB  (100.0%)
2021-04-30T19:00:39.023+0800	no indexes to restore
2021-04-30T19:00:39.023+0800	finished restoring db4.test1 (1000000 documents, 0 failures)
2021-04-30T19:00:40.386+0800	[########################]  db4.db4  137MB/137MB  (100.0%)
2021-04-30T19:00:40.386+0800	no indexes to restore
2021-04-30T19:00:40.386+0800	finished restoring db4.db4 (1313657 documents, 0 failures)
2021-04-30T19:00:41.097+0800	[####....................]  db4.rcmd_1_tag_li_liao  684MB/3.32GB  (20.1%)
2021-04-30T19:00:44.097+0800	[#####...................]  db4.rcmd_1_tag_li_liao  760MB/3.32GB  (22.3%)
2021-04-30T19:00:47.097+0800	[#####...................]  db4.rcmd_1_tag_li_liao  836MB/3.32GB  (24.6%)
2021-04-30T19:00:50.098+0800	[######..................]  db4.rcmd_1_tag_li_liao  906MB/3.32GB  (26.6%)
2021-04-30T19:00:53.098+0800	[#######.................]  db4.rcmd_1_tag_li_liao  994MB/3.32GB  (29.2%)
2021-04-30T19:00:56.098+0800	[#######.................]  db4.rcmd_1_tag_li_liao  1.03GB/3.32GB  (31.0%)
2021-04-30T19:00:59.098+0800	[########................]  db4.rcmd_1_tag_li_liao  1.11GB/3.32GB  (33.3%)
2021-04-30T19:01:02.097+0800	[########................]  db4.rcmd_1_tag_li_liao  1.18GB/3.32GB  (35.5%)
2021-04-30T19:01:05.101+0800	[#########...............]  db4.rcmd_1_tag_li_liao  1.26GB/3.32GB  (38.0%)
2021-04-30T19:01:08.097+0800	[#########...............]  db4.rcmd_1_tag_li_liao  1.32GB/3.32GB  (39.7%)
2021-04-30T19:01:11.100+0800	[#########...............]  db4.rcmd_1_tag_li_liao  1.37GB/3.32GB  (41.1%)
2021-04-30T19:01:14.098+0800	[##########..............]  db4.rcmd_1_tag_li_liao  1.43GB/3.32GB  (43.2%)
2021-04-30T19:01:17.097+0800	[##########..............]  db4.rcmd_1_tag_li_liao  1.50GB/3.32GB  (45.0%)
2021-04-30T19:01:20.098+0800	[###########.............]  db4.rcmd_1_tag_li_liao  1.54GB/3.32GB  (46.3%)
2021-04-30T19:01:23.098+0800	[###########.............]  db4.rcmd_1_tag_li_liao  1.58GB/3.32GB  (47.6%)
2021-04-30T19:01:26.098+0800	[###########.............]  db4.rcmd_1_tag_li_liao  1.64GB/3.32GB  (49.3%)
2021-04-30T19:01:29.097+0800	[############............]  db4.rcmd_1_tag_li_liao  1.71GB/3.32GB  (51.4%)
2021-04-30T19:01:32.097+0800	[############............]  db4.rcmd_1_tag_li_liao  1.77GB/3.32GB  (53.2%)
2021-04-30T19:01:35.098+0800	[#############...........]  db4.rcmd_1_tag_li_liao  1.85GB/3.32GB  (55.7%)
2021-04-30T19:01:38.097+0800	[#############...........]  db4.rcmd_1_tag_li_liao  1.90GB/3.32GB  (57.2%)
2021-04-30T19:01:41.097+0800	[##############..........]  db4.rcmd_1_tag_li_liao  1.98GB/3.32GB  (59.5%)
2021-04-30T19:01:44.827+0800	[##############..........]  db4.rcmd_1_tag_li_liao  2.00GB/3.32GB  (60.3%)
2021-04-30T19:01:47.097+0800	[##############..........]  db4.rcmd_1_tag_li_liao  2.05GB/3.32GB  (61.8%)
2021-04-30T19:01:50.098+0800	[###############.........]  db4.rcmd_1_tag_li_liao  2.12GB/3.32GB  (63.9%)
2021-04-30T19:01:53.097+0800	[###############.........]  db4.rcmd_1_tag_li_liao  2.19GB/3.32GB  (65.9%)
2021-04-30T19:01:56.097+0800	[################........]  db4.rcmd_1_tag_li_liao  2.26GB/3.32GB  (67.9%)
2021-04-30T19:01:59.099+0800	[################........]  db4.rcmd_1_tag_li_liao  2.32GB/3.32GB  (69.8%)
2021-04-30T19:02:02.098+0800	[#################.......]  db4.rcmd_1_tag_li_liao  2.39GB/3.32GB  (72.0%)
2021-04-30T19:02:05.097+0800	[#################.......]  db4.rcmd_1_tag_li_liao  2.47GB/3.32GB  (74.4%)
2021-04-30T19:02:08.097+0800	[##################......]  db4.rcmd_1_tag_li_liao  2.52GB/3.32GB  (76.0%)
2021-04-30T19:02:11.097+0800	[##################......]  db4.rcmd_1_tag_li_liao  2.59GB/3.32GB  (77.8%)
2021-04-30T19:02:14.097+0800	[###################.....]  db4.rcmd_1_tag_li_liao  2.66GB/3.32GB  (80.0%)
2021-04-30T19:02:17.097+0800	[###################.....]  db4.rcmd_1_tag_li_liao  2.72GB/3.32GB  (81.9%)
2021-04-30T19:02:20.097+0800	[####################....]  db4.rcmd_1_tag_li_liao  2.78GB/3.32GB  (83.7%)
2021-04-30T19:02:23.097+0800	[####################....]  db4.rcmd_1_tag_li_liao  2.85GB/3.32GB  (85.7%)
2021-04-30T19:02:26.098+0800	[#####################...]  db4.rcmd_1_tag_li_liao  2.94GB/3.32GB  (88.4%)
2021-04-30T19:02:29.097+0800	[#####################...]  db4.rcmd_1_tag_li_liao  3.00GB/3.32GB  (90.4%)
2021-04-30T19:02:32.097+0800	[######################..]  db4.rcmd_1_tag_li_liao  3.06GB/3.32GB  (92.1%)
2021-04-30T19:02:35.099+0800	[######################..]  db4.rcmd_1_tag_li_liao  3.12GB/3.32GB  (93.9%)
2021-04-30T19:02:38.097+0800	[######################..]  db4.rcmd_1_tag_li_liao  3.15GB/3.32GB  (95.0%)
2021-04-30T19:02:41.098+0800	[#######################.]  db4.rcmd_1_tag_li_liao  3.21GB/3.32GB  (96.7%)
2021-04-30T19:02:44.167+0800	[#######################.]  db4.rcmd_1_tag_li_liao  3.26GB/3.32GB  (98.0%)
2021-04-30T19:02:47.097+0800	[#######################.]  db4.rcmd_1_tag_li_liao  3.32GB/3.32GB  (99.9%)
2021-04-30T19:02:47.392+0800	[########################]  db4.rcmd_1_tag_li_liao  3.32GB/3.32GB  (100.0%)
2021-04-30T19:02:47.393+0800	no indexes to restore
2021-04-30T19:02:47.393+0800	finished restoring db4.rcmd_1_tag_li_liao (379143 documents, 0 failures)
2021-04-30T19:02:47.393+0800	restoring users from /mongodb/backup/backup/admin/system.users.bson
2021-04-30T19:02:50.655+0800	admin.tempusers  2.05KB
2021-04-30T19:02:50.655+0800	admin.tempusers  2.05KB
2021-04-30T19:02:51.905+0800	replaying oplog
2021-04-30T19:02:53.097+0800	oplog  483KB
2021-04-30T19:02:56.097+0800	oplog  20.2MB
2021-04-30T19:02:59.100+0800	oplog  36.9MB
2021-04-30T19:03:02.097+0800	oplog  50.1MB
2021-04-30T19:03:05.098+0800	oplog  69.0MB
2021-04-30T19:03:08.097+0800	oplog  90.6MB
2021-04-30T19:03:11.097+0800	oplog  124MB
2021-04-30T19:03:14.097+0800	oplog  159MB
2021-04-30T19:03:17.098+0800	oplog  185MB
2021-04-30T19:03:20.097+0800	oplog  219MB
2021-04-30T19:03:23.098+0800	oplog  256MB
2021-04-30T19:03:26.097+0800	oplog  290MB
2021-04-30T19:03:29.097+0800	oplog  323MB
2021-04-30T19:03:32.097+0800	oplog  357MB
2021-04-30T19:03:35.097+0800	oplog  391MB
2021-04-30T19:03:38.098+0800	oplog  427MB
2021-04-30T19:03:41.098+0800	oplog  464MB
2021-04-30T19:03:44.097+0800	oplog  501MB
2021-04-30T19:03:47.097+0800	oplog  544MB
2021-04-30T19:03:50.098+0800	oplog  580MB
2021-04-30T19:03:53.098+0800	oplog  619MB
2021-04-30T19:03:56.098+0800	oplog  650MB
2021-04-30T19:03:59.099+0800	oplog  687MB
2021-04-30T19:04:02.097+0800	oplog  722MB
2021-04-30T19:04:05.097+0800	oplog  758MB
2021-04-30T19:04:08.098+0800	oplog  795MB
2021-04-30T19:04:11.097+0800	oplog  826MB
2021-04-30T19:04:14.098+0800	oplog  858MB
2021-04-30T19:04:17.102+0800	oplog  893MB
2021-04-30T19:04:20.097+0800	oplog  929MB
2021-04-30T19:04:23.098+0800	oplog  968MB
2021-04-30T19:04:26.098+0800	oplog  1001MB
2021-04-30T19:04:29.097+0800	oplog  1002MB
2021-04-30T19:04:32.097+0800	oplog  1003MB
2021-04-30T19:04:35.097+0800	oplog  1004MB
2021-04-30T19:04:38.097+0800	oplog  1004MB
2021-04-30T19:04:38.597+0800	applied 130029 oplog entries
2021-04-30T19:04:38.597+0800	oplog  1005MB
2021-04-30T19:04:38.614+0800	2766634 document(s) restored successfully. 0 document(s) failed to restore.

检查恢复结果

> show dbs
admin   0.000GB
config  0.000GB
db3     0.000GB
db4     3.355GB
db5     0.000GB
local   0.000GB
ycsb    0.060GB
ycsb1   0.041GB
> 
> 
> use db5
switched to db db5
> 
> db.test.count()
100

可以看到我们前期插入的100记录以及恢复成功。

到此这篇关于MongoDB利用oplog恢复数据介绍的文章就介绍到这了,更多相关oplog数据恢复内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木！

MongoDB误操作后使用oplog恢复数据

- Author -

那海蓝蓝

- Original Sources -

声明：登载此文出于传递更多信息之目的，并不意味着赞同其观点或证实其描述。

MongoDB 相关文章推荐

MongoDB数据库常用的10条操作命令

Jun 18 MongoDB

浅析MongoDB之安全认证

Jun 26 MongoDB

MongoDB安装使用并实现Python操作数据库

Jun 28 MongoDB

mongodb的安装和开机自启动详细讲解

Aug 02 MongoDB

阿里云服务器部署mongodb的详细过程

Sep 04 MongoDB

MongoDB日志切割的三种方式总结

Sep 15 MongoDB

关于CentOS 8 搭建MongoDB4.4分片集群的问题

Oct 24 MongoDB

MongoDB连接数据库并创建数据等使用方法

Nov 27 MongoDB

SpringBoot集成MongoDB实现文件上传的步骤

Apr 18 MongoDB

MongoDB数据库之添删改查

Apr 26 MongoDB

mongoDB数据库索引快速入门指南

MongoDB数据库部署环境准备及使用介绍

一次线上mongo慢查询问题排查处理记录

Mar 18 #MongoDB

SpringBoot 整合mongoDB并自定义连接池的示例代码

Feb 28 #MongoDB

剖析后OpLog订阅MongoDB的数据变更就没那么难了

MongoDB使用场景总结

SpringBoot系列之MongoDB Aggregations用法详解

MongoDB误操作后使用oplog恢复数据

目录

数据全备

模拟故障

写入数据

模拟误操作

恢复步骤

备份oplog

解析oplog

将oplog备份和全备复制到standalone机

查找误操作时间点

进行数据恢复

检查恢复结果