如何编写正则表达式从stderr日志文件中提取hadoopMR计数器数据，
如何查找具有\t\t线对数据所有\t线，我编写了一个规则（re.findall(r'(\t[a-zA-Z\s]+)\n(.*?)\n\t\w+', text, re.S|re.M)），但它不正确
这是stderr日志示例
23/01/16 14:26:13 INFO mortbay.log: Conf is not init.
23/01/16 14:26:14 INFO mapreduce.Job: Counters: 246
File System Counters
FILE: Number of bytes read=104971581500
FILE: Number of bytes written=287906526786
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=758223470025
HDFS: Number of bytes written=97994290043
HDFS: Number of read operations=24275
HDFS: Number of large read operations=0
HDFS: Number of write operations=2000
VIEWFS: Number of bytes read=0
VIEWFS: Number of bytes written=0
VIEWFS: Number of read operations=0
VIEWFS: Number of large read operations=0
VIEWFS: Number of write operations=0
Job Counters
Killed map tasks=3
Killed reduce tasks=2
Launched map tasks=6427
Launched reduce tasks=1002
Other local map tasks=33
Data-local map tasks=3746
Rack-local map tasks=2648
Total time spent by all maps in occupied slots (ms)=358061940
Total time spent by all reduces in occupied slots (ms)=858021936
Total time spent by all map tasks (ms)=119353980
Total time spent by all reduce tasks (ms)=107252742
Total vcore-milliseconds taken by all map tasks=119353980
Total vcore-milliseconds taken by all reduce tasks=107252742
Total megabyte-milliseconds taken by all map tasks=305546188800
Total megabyte-milliseconds taken by all reduce tasks=878614462464
Map-Reduce Framework
Map input records=30951997
Map output records=30951997
Shuffled Maps =6425000
Failed Shuffles=46
Merged Map outputs=6425000
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=0
23/01/16 14:26:14 INFO streaming.StreamJob: Output directory:
+ [[ 0 -ne 0 ]]
+ exit 0
1条答案
按热度按时间omqzjyyz1#
您可以匹配以单个制表符开头的行，后跟以2个制表符和一个非空格字符开头的一行或多行：
您也只能使用
re.M参见regex101 demo。
如果匹配2个制表符后行的其余部分也可以为空，则只能为
.*参见另一个regex101 demo。