首页
所有分类
TAGS
文字工具 EMOJI BIBLE
数据库 MongoDB

MongoDB误操作后使用oplog恢复数据

Posted in MongoDB onApril 11, 2022

当我们对数据出现误操作的时候，可以利用oplog恢复数据。

使用前提：

  • 1、环境是副本集
  • 2、必须有全备
  • 2、全备后oplog没有被覆盖

数据全备

mongodump -h 172.16.254.133 --port 27017 -o /mongodb/backup/backup

模拟故障

写入数据

handong1:PRIMARY> for (var i = 1; i <= 100; i++) {
...    db.test.insert( { id : i , name: "handong" , name1:"handong", name2:"handong", name3:"handong"} )
... }
WriteResult({ "nInserted" : 1 })
handong1:PRIMARY> 
handong1:PRIMARY> db.test.count()
100

模拟误操作

handong1:PRIMARY> db.test.remove({})
WriteResult({ "nRemoved" : 100 })
handong1:PRIMARY> db.test.count()
0

所有文档被误删除。

恢复步骤

备份oplog

mongodump -h 172.16.254.133 --port 27017 -d local -c oplog.rs -o /mongodb/backup
2021-04-30T18:32:29.077+0800	writing local.oplog.rs to /mongodb/backup/local/oplog.rs.bson
2021-04-30T18:32:32.039+0800	local.oplog.rs  7108
2021-04-30T18:32:35.038+0800	local.oplog.rs  17912
2021-04-30T18:32:38.041+0800	local.oplog.rs  28226
2021-04-30T18:32:41.039+0800	local.oplog.rs  38642
2021-04-30T18:32:44.042+0800	local.oplog.rs  50679
2021-04-30T18:32:47.040+0800	local.oplog.rs  64001
2021-04-30T18:32:50.040+0800	local.oplog.rs  77265
2021-04-30T18:32:53.038+0800	local.oplog.rs  89739
2021-04-30T18:32:56.038+0800	local.oplog.rs  102449
2021-04-30T18:32:57.697+0800	local.oplog.rs  132459
2021-04-30T18:32:57.697+0800	done dumping local.oplog.rs (132459 documents)

解析oplog

bsondump /mongodb/backup/local/oplog.rs.bson  > /mongodb/backup/local/local.log
2021-04-30T18:34:27.612+0800	132459 objects found

将oplog备份和全备复制到standalone机

scp -r backup/ 172.16.254.130:/mongodb/backup/
scp -r local/ 172.16.254.130:/mongodb/backup/backup

查找误操作时间点

通过查看解析完的日志local.log发现误操作的时间点在1619778429。

进行数据恢复

mongorestore -h 172.16.254.130 --port 27017 --oplogReplay --oplogLimit 1619778429:1 /mongodb/backup/backup
2021-04-30T19:00:11.099+0800	preparing collections to restore from
2021-04-30T19:00:11.100+0800	don't know what to do with file "/mongodb/backup/backup/local/111.log", skipping...
2021-04-30T19:00:11.100+0800	don't know what to do with file "/mongodb/backup/backup/local/local.log", skipping...
2021-04-30T19:00:11.116+0800	reading metadata for db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.metadata.json
2021-04-30T19:00:11.117+0800	reading metadata for ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.metadata.json
2021-04-30T19:00:11.119+0800	reading metadata for db3.db3 from /mongodb/backup/backup/db3/db3.metadata.json
2021-04-30T19:00:11.119+0800	reading metadata for ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.metadata.json
2021-04-30T19:00:11.170+0800	restoring ycsb.usertable from /mongodb/backup/backup/ycsb/usertable.bson
2021-04-30T19:00:11.187+0800	restoring ycsb1.usertable from /mongodb/backup/backup/ycsb1/usertable.bson
2021-04-30T19:00:11.391+0800	restoring db4.rcmd_1_tag_li_liao from /mongodb/backup/backup/db4/rcmd_1_tag_li_liao.bson
2021-04-30T19:00:11.580+0800	restoring db3.db3 from /mongodb/backup/backup/db3/db3.bson
2021-04-30T19:00:11.661+0800	no indexes to restore
2021-04-30T19:00:11.661+0800	finished restoring db3.db3 (6 documents, 0 failures)
2021-04-30T19:00:11.662+0800	reading metadata for db5.test from /mongodb/backup/backup/db5/test.metadata.json
2021-04-30T19:00:12.545+0800	restoring db5.test from /mongodb/backup/backup/db5/test.bson
2021-04-30T19:00:12.548+0800	no indexes to restore
2021-04-30T19:00:12.548+0800	finished restoring db5.test (0 documents, 0 failures)
2021-04-30T19:00:12.548+0800	reading metadata for db4.db4 from /mongodb/backup/backup/db4/db4.metadata.json
2021-04-30T19:00:13.784+0800	no indexes to restore
2021-04-30T19:00:13.784+0800	finished restoring ycsb1.usertable (30370 documents, 0 failures)
2021-04-30T19:00:13.785+0800	reading metadata for db4.test1 from /mongodb/backup/backup/db4/test1.metadata.json
2021-04-30T19:00:14.099+0800	[###################.....]          ycsb.usertable  47.8MB/58.9MB  (81.2%)
2021-04-30T19:00:14.099+0800	[........................]  db4.rcmd_1_tag_li_liao  79.8MB/3.32GB   (2.3%)
2021-04-30T19:00:14.099+0800	
2021-04-30T19:00:14.843+0800	[########################]  ycsb.usertable  58.9MB/58.9MB  (100.0%)
2021-04-30T19:00:14.843+0800	no indexes to restore
2021-04-30T19:00:14.843+0800	finished restoring ycsb.usertable (43458 documents, 0 failures)
2021-04-30T19:00:15.339+0800	restoring db4.db4 from /mongodb/backup/backup/db4/db4.bson
2021-04-30T19:00:17.077+0800	restoring db4.test1 from /mongodb/backup/backup/db4/test1.bson
2021-04-30T19:00:17.103+0800	[#.......................]  db4.rcmd_1_tag_li_liao  167MB/3.32GB  (4.9%)
2021-04-30T19:00:17.104+0800	[#.......................]                 db4.db4  8.11MB/137MB  (5.9%)
2021-04-30T19:00:17.104+0800	[........................]               db4.test1      0B/104MB  (0.0%)
2021-04-30T19:00:17.104+0800	
2021-04-30T19:00:20.099+0800	[#.......................]  db4.rcmd_1_tag_li_liao  204MB/3.32GB   (6.0%)
2021-04-30T19:00:20.099+0800	[####....................]                 db4.db4  24.5MB/137MB  (18.0%)
2021-04-30T19:00:20.099+0800	[#.......................]               db4.test1  4.47MB/104MB   (4.3%)
2021-04-30T19:00:20.099+0800	
2021-04-30T19:00:23.099+0800	[#.......................]  db4.rcmd_1_tag_li_liao  272MB/3.32GB   (8.0%)
2021-04-30T19:00:23.099+0800	[######..................]                 db4.db4  39.7MB/137MB  (29.1%)
2021-04-30T19:00:23.099+0800	[####....................]               db4.test1  20.1MB/104MB  (19.3%)
2021-04-30T19:00:23.099+0800	
2021-04-30T19:00:26.102+0800	[##......................]  db4.rcmd_1_tag_li_liao  355MB/3.32GB  (10.4%)
2021-04-30T19:00:26.102+0800	[##########..............]                 db4.db4  58.0MB/137MB  (42.5%)
2021-04-30T19:00:26.102+0800	[########................]               db4.test1  38.1MB/104MB  (36.7%)
2021-04-30T19:00:26.102+0800	
2021-04-30T19:00:29.098+0800	[##......................]  db4.rcmd_1_tag_li_liao  401MB/3.32GB  (11.8%)
2021-04-30T19:00:29.098+0800	[############............]                 db4.db4  73.1MB/137MB  (53.5%)
2021-04-30T19:00:29.098+0800	[###########.............]               db4.test1  51.8MB/104MB  (49.8%)
2021-04-30T19:00:29.098+0800	
2021-04-30T19:00:32.097+0800	[###.....................]  db4.rcmd_1_tag_li_liao  494MB/3.32GB  (14.5%)
2021-04-30T19:00:32.097+0800	[###############.........]                 db4.db4  90.8MB/137MB  (66.5%)
2021-04-30T19:00:32.097+0800	[###############.........]               db4.test1  67.3MB/104MB  (64.7%)
2021-04-30T19:00:32.097+0800	
2021-04-30T19:00:35.100+0800	[###.....................]  db4.rcmd_1_tag_li_liao  556MB/3.32GB  (16.3%)
2021-04-30T19:00:35.100+0800	[###################.....]                 db4.db4   110MB/137MB  (80.5%)
2021-04-30T19:00:35.100+0800	[###################.....]               db4.test1  86.1MB/104MB  (82.8%)
2021-04-30T19:00:35.100+0800	
2021-04-30T19:00:38.097+0800	[####....................]  db4.rcmd_1_tag_li_liao  620MB/3.32GB  (18.2%)
2021-04-30T19:00:38.097+0800	[#####################...]                 db4.db4   124MB/137MB  (91.1%)
2021-04-30T19:00:38.097+0800	[#######################.]               db4.test1   101MB/104MB  (96.7%)
2021-04-30T19:00:38.097+0800	
2021-04-30T19:00:39.023+0800	[########################]  db4.test1  104MB/104MB  (100.0%)
2021-04-30T19:00:39.023+0800	no indexes to restore
2021-04-30T19:00:39.023+0800	finished restoring db4.test1 (1000000 documents, 0 failures)
2021-04-30T19:00:40.386+0800	[########################]  db4.db4  137MB/137MB  (100.0%)
2021-04-30T19:00:40.386+0800	no indexes to restore
2021-04-30T19:00:40.386+0800	finished restoring db4.db4 (1313657 documents, 0 failures)
2021-04-30T19:00:41.097+0800	[####....................]  db4.rcmd_1_tag_li_liao  684MB/3.32GB  (20.1%)
2021-04-30T19:00:44.097+0800	[#####...................]  db4.rcmd_1_tag_li_liao  760MB/3.32GB  (22.3%)
2021-04-30T19:00:47.097+0800	[#####...................]  db4.rcmd_1_tag_li_liao  836MB/3.32GB  (24.6%)
2021-04-30T19:00:50.098+0800	[######..................]  db4.rcmd_1_tag_li_liao  906MB/3.32GB  (26.6%)
2021-04-30T19:00:53.098+0800	[#######.................]  db4.rcmd_1_tag_li_liao  994MB/3.32GB  (29.2%)
2021-04-30T19:00:56.098+0800	[#######.................]  db4.rcmd_1_tag_li_liao  1.03GB/3.32GB  (31.0%)
2021-04-30T19:00:59.098+0800	[########................]  db4.rcmd_1_tag_li_liao  1.11GB/3.32GB  (33.3%)
2021-04-30T19:01:02.097+0800	[########................]  db4.rcmd_1_tag_li_liao  1.18GB/3.32GB  (35.5%)
2021-04-30T19:01:05.101+0800	[#########...............]  db4.rcmd_1_tag_li_liao  1.26GB/3.32GB  (38.0%)
2021-04-30T19:01:08.097+0800	[#########...............]  db4.rcmd_1_tag_li_liao  1.32GB/3.32GB  (39.7%)
2021-04-30T19:01:11.100+0800	[#########...............]  db4.rcmd_1_tag_li_liao  1.37GB/3.32GB  (41.1%)
2021-04-30T19:01:14.098+0800	[##########..............]  db4.rcmd_1_tag_li_liao  1.43GB/3.32GB  (43.2%)
2021-04-30T19:01:17.097+0800	[##########..............]  db4.rcmd_1_tag_li_liao  1.50GB/3.32GB  (45.0%)
2021-04-30T19:01:20.098+0800	[###########.............]  db4.rcmd_1_tag_li_liao  1.54GB/3.32GB  (46.3%)
2021-04-30T19:01:23.098+0800	[###########.............]  db4.rcmd_1_tag_li_liao  1.58GB/3.32GB  (47.6%)
2021-04-30T19:01:26.098+0800	[###########.............]  db4.rcmd_1_tag_li_liao  1.64GB/3.32GB  (49.3%)
2021-04-30T19:01:29.097+0800	[############............]  db4.rcmd_1_tag_li_liao  1.71GB/3.32GB  (51.4%)
2021-04-30T19:01:32.097+0800	[############............]  db4.rcmd_1_tag_li_liao  1.77GB/3.32GB  (53.2%)
2021-04-30T19:01:35.098+0800	[#############...........]  db4.rcmd_1_tag_li_liao  1.85GB/3.32GB  (55.7%)
2021-04-30T19:01:38.097+0800	[#############...........]  db4.rcmd_1_tag_li_liao  1.90GB/3.32GB  (57.2%)
2021-04-30T19:01:41.097+0800	[##############..........]  db4.rcmd_1_tag_li_liao  1.98GB/3.32GB  (59.5%)
2021-04-30T19:01:44.827+0800	[##############..........]  db4.rcmd_1_tag_li_liao  2.00GB/3.32GB  (60.3%)
2021-04-30T19:01:47.097+0800	[##############..........]  db4.rcmd_1_tag_li_liao  2.05GB/3.32GB  (61.8%)
2021-04-30T19:01:50.098+0800	[###############.........]  db4.rcmd_1_tag_li_liao  2.12GB/3.32GB  (63.9%)
2021-04-30T19:01:53.097+0800	[###############.........]  db4.rcmd_1_tag_li_liao  2.19GB/3.32GB  (65.9%)
2021-04-30T19:01:56.097+0800	[################........]  db4.rcmd_1_tag_li_liao  2.26GB/3.32GB  (67.9%)
2021-04-30T19:01:59.099+0800	[################........]  db4.rcmd_1_tag_li_liao  2.32GB/3.32GB  (69.8%)
2021-04-30T19:02:02.098+0800	[#################.......]  db4.rcmd_1_tag_li_liao  2.39GB/3.32GB  (72.0%)
2021-04-30T19:02:05.097+0800	[#################.......]  db4.rcmd_1_tag_li_liao  2.47GB/3.32GB  (74.4%)
2021-04-30T19:02:08.097+0800	[##################......]  db4.rcmd_1_tag_li_liao  2.52GB/3.32GB  (76.0%)
2021-04-30T19:02:11.097+0800	[##################......]  db4.rcmd_1_tag_li_liao  2.59GB/3.32GB  (77.8%)
2021-04-30T19:02:14.097+0800	[###################.....]  db4.rcmd_1_tag_li_liao  2.66GB/3.32GB  (80.0%)
2021-04-30T19:02:17.097+0800	[###################.....]  db4.rcmd_1_tag_li_liao  2.72GB/3.32GB  (81.9%)
2021-04-30T19:02:20.097+0800	[####################....]  db4.rcmd_1_tag_li_liao  2.78GB/3.32GB  (83.7%)
2021-04-30T19:02:23.097+0800	[####################....]  db4.rcmd_1_tag_li_liao  2.85GB/3.32GB  (85.7%)
2021-04-30T19:02:26.098+0800	[#####################...]  db4.rcmd_1_tag_li_liao  2.94GB/3.32GB  (88.4%)
2021-04-30T19:02:29.097+0800	[#####################...]  db4.rcmd_1_tag_li_liao  3.00GB/3.32GB  (90.4%)
2021-04-30T19:02:32.097+0800	[######################..]  db4.rcmd_1_tag_li_liao  3.06GB/3.32GB  (92.1%)
2021-04-30T19:02:35.099+0800	[######################..]  db4.rcmd_1_tag_li_liao  3.12GB/3.32GB  (93.9%)
2021-04-30T19:02:38.097+0800	[######################..]  db4.rcmd_1_tag_li_liao  3.15GB/3.32GB  (95.0%)
2021-04-30T19:02:41.098+0800	[#######################.]  db4.rcmd_1_tag_li_liao  3.21GB/3.32GB  (96.7%)
2021-04-30T19:02:44.167+0800	[#######################.]  db4.rcmd_1_tag_li_liao  3.26GB/3.32GB  (98.0%)
2021-04-30T19:02:47.097+0800	[#######################.]  db4.rcmd_1_tag_li_liao  3.32GB/3.32GB  (99.9%)
2021-04-30T19:02:47.392+0800	[########################]  db4.rcmd_1_tag_li_liao  3.32GB/3.32GB  (100.0%)
2021-04-30T19:02:47.393+0800	no indexes to restore
2021-04-30T19:02:47.393+0800	finished restoring db4.rcmd_1_tag_li_liao (379143 documents, 0 failures)
2021-04-30T19:02:47.393+0800	restoring users from /mongodb/backup/backup/admin/system.users.bson
2021-04-30T19:02:50.655+0800	admin.tempusers  2.05KB
2021-04-30T19:02:50.655+0800	admin.tempusers  2.05KB
2021-04-30T19:02:51.905+0800	replaying oplog
2021-04-30T19:02:53.097+0800	oplog  483KB
2021-04-30T19:02:56.097+0800	oplog  20.2MB
2021-04-30T19:02:59.100+0800	oplog  36.9MB
2021-04-30T19:03:02.097+0800	oplog  50.1MB
2021-04-30T19:03:05.098+0800	oplog  69.0MB
2021-04-30T19:03:08.097+0800	oplog  90.6MB
2021-04-30T19:03:11.097+0800	oplog  124MB
2021-04-30T19:03:14.097+0800	oplog  159MB
2021-04-30T19:03:17.098+0800	oplog  185MB
2021-04-30T19:03:20.097+0800	oplog  219MB
2021-04-30T19:03:23.098+0800	oplog  256MB
2021-04-30T19:03:26.097+0800	oplog  290MB
2021-04-30T19:03:29.097+0800	oplog  323MB
2021-04-30T19:03:32.097+0800	oplog  357MB
2021-04-30T19:03:35.097+0800	oplog  391MB
2021-04-30T19:03:38.098+0800	oplog  427MB
2021-04-30T19:03:41.098+0800	oplog  464MB
2021-04-30T19:03:44.097+0800	oplog  501MB
2021-04-30T19:03:47.097+0800	oplog  544MB
2021-04-30T19:03:50.098+0800	oplog  580MB
2021-04-30T19:03:53.098+0800	oplog  619MB
2021-04-30T19:03:56.098+0800	oplog  650MB
2021-04-30T19:03:59.099+0800	oplog  687MB
2021-04-30T19:04:02.097+0800	oplog  722MB
2021-04-30T19:04:05.097+0800	oplog  758MB
2021-04-30T19:04:08.098+0800	oplog  795MB
2021-04-30T19:04:11.097+0800	oplog  826MB
2021-04-30T19:04:14.098+0800	oplog  858MB
2021-04-30T19:04:17.102+0800	oplog  893MB
2021-04-30T19:04:20.097+0800	oplog  929MB
2021-04-30T19:04:23.098+0800	oplog  968MB
2021-04-30T19:04:26.098+0800	oplog  1001MB
2021-04-30T19:04:29.097+0800	oplog  1002MB
2021-04-30T19:04:32.097+0800	oplog  1003MB
2021-04-30T19:04:35.097+0800	oplog  1004MB
2021-04-30T19:04:38.097+0800	oplog  1004MB
2021-04-30T19:04:38.597+0800	applied 130029 oplog entries
2021-04-30T19:04:38.597+0800	oplog  1005MB
2021-04-30T19:04:38.614+0800	2766634 document(s) restored successfully. 0 document(s) failed to restore.

检查恢复结果

> show dbs
admin   0.000GB
config  0.000GB
db3     0.000GB
db4     3.355GB
db5     0.000GB
local   0.000GB
ycsb    0.060GB
ycsb1   0.041GB
> 
> 
> use db5
switched to db db5
> 
> db.test.count()
100

可以看到我们前期插入的100记录以及恢复成功。

到此这篇关于MongoDB利用oplog恢复数据介绍的文章就介绍到这了,更多相关oplog数据恢复内容请搜索三水点靠木以前的文章或继续浏览下面的相关文章希望大家以后多多支持三水点靠木！

MongoDB误操作后使用oplog恢复数据

- Author -

那海蓝蓝

- Original Sources -

声明：登载此文出于传递更多信息之目的，并不意味着赞同其观点或证实其描述。

MongoDB 相关文章推荐
详解MongoDB的条件查询和排序
Jun 23 MongoDB
SpringBoot整合MongoDB的实现步骤
Jun 23 MongoDB
MongoDB日志切割的三种方式总结
Sep 15 MongoDB
springboot + mongodb 通过经纬度坐标匹配平面区域的方法
Nov 01 MongoDB
MongoDB连接数据库并创建数据等使用方法
Nov 27 MongoDB
MongoDB支持的数据类型
Apr 11 MongoDB
MongoDB支持的索引类型
Apr 11 MongoDB
MongoDB修改oplog大小的四种方法
Apr 11 MongoDB
Mongodb 迁移数据块的流程介绍分析
Apr 18 MongoDB
NoSQL优缺点与ＭongoDB数据库简介
Jun 05 MongoDB
mongoDB数据库索引快速入门指南
MongoDB数据库部署环境准备及使用介绍
一次线上mongo慢查询问题排查处理记录
Mar 18 #MongoDB
SpringBoot 整合mongoDB并自定义连接池的示例代码
Feb 28 #MongoDB
剖析后OpLog订阅MongoDB的数据变更就没那么难了
MongoDB使用场景总结
SpringBoot系列之MongoDB Aggregations用法详解
V Rising(1) Android Gradle(1) maven(2) Spring Security(2) 内置函数(1) 记事本(1) Tablespace(1) 自定义函数(1) procedure(1) Navicat(1)
You might like
咖啡冲泡指南 咖啡有哪些制作方式 单品咖啡 意式咖啡
2021/03/06 冲泡冲煮
树型结构列出指定目录里所有文件的PHP类
2006/10/09 PHP
php 正确解码javascript中通过escape编码后的字符
2010/01/28 PHP
php获取发送给用户的header信息的方法
2015/03/16 PHP
ExtJS 简介 让你知道extjs是什么
2008/12/29 Javascript
JavaScript 学习历程和心得分享
2010/12/12 Javascript
JS实现在网页中弹出一个输入框的方法
2015/03/03 Javascript
javascript实现超炫的向上滑行菜单实例
2015/08/03 Javascript
iOS和Android用同一个二维码实现跳转下载链接的方法
2016/09/28 Javascript
javascript简单进制转换实现方法
2016/11/24 Javascript
利用Js的console对象,在控制台打印调式信息测试Js的实现
2016/11/26 Javascript
jQuery插件FusionCharts绘制2D双折线图效果示例【附demo源码】
2017/04/14 jQuery
VUE axios上传图片到七牛的实例代码
2017/07/28 Javascript
浅谈Vue响应式(数组变异方法)
2018/05/07 Javascript
js实现计时器秒表功能
2019/12/16 Javascript
JS实现网页时钟特效
2020/03/25 Javascript
Vue点击切换Class变化,实现Active当前样式操作
2020/07/17 Javascript
[53:52]EG vs VGJ.T 2018国际邀请赛小组赛BO2 第一场 8.16
2018/08/17 DOTA
[32:07]完美世界DOTA2联赛PWL S3 LBZS vs Rebirth 第一场 12.16
2020/12/17 DOTA
python3使用scrapy生成csv文件代码示例
2017/12/28 Python
Python中反射和描述器总结
2018/09/23 Python
Python时间和字符串转换操作实例分析
2019/03/16 Python
python中时间、日期、时间戳的转换的实现方法
2019/07/06 Python
python读取指定字节长度的文本方法
2019/08/27 Python
python 通过pip freeze、dowload打离线包及自动安装的过程详解(适用于保密的离线环境
2020/12/14 Python
python实现跨年表白神器--你值得拥有
2021/01/04 Python
一款简洁的纯css3代码实现的动画导航
2014/10/31 HTML / CSS
行政总经理岗位职责
2013/12/05 职场文书
全国法院系统开展党的群众路线教育实践活动综述（全文）
2014/10/25 职场文书
承诺书范本
2015/01/21 职场文书
2015毕业寄语大全
2015/02/26 职场文书
2015年教研员工作总结
2015/05/26 职场文书
给朋友的赠语
2015/06/23 职场文书
2016暑期社会实践心得体会范文
2016/01/14 职场文书
2016年度基层党建工作公开承诺书
2016/03/25 职场文书
Android自定义scrollview实现回弹效果
2022/04/01 Java/Android