LINUX.ORG.RU
Форум — Development  

Почистить лог от дублей

 


1

2

Уважаемые знатоки, прошу помощи. Есть файл, пример: 

2008000286248 | card exist(41574) | anketa exist(10841) | Sex present | Bday is present | Phone is present | no correct E-mail  | Source is present
2010000312728 | card exist(142748) | anketa exist(24628) | Sex present | Bday is present | Phone is present | E-mail is present | upadte Source (AZS)
2010000312728 | card exist(72032) | anketa exist(24628) | Sex present | Bday is present | Phone is present | no correct E-mail  | Source is present
2010000312728 | card exist(142748) | anketa exist(24628) | Sex present | Bday is present | Phone is present | E-mail is present | upadte Source (AZS)
2008000134624 | card exist(52935) | anketa exist(10842) | Sex present | Bday is present | Phone is present | no correct E-mail  | upadte Source (AZS)
2010000002810 | card exist(72032) | anketa exist(10843) | Sex present | Bday is present | Phone is present | no correct E-mail  | upadte Source (AZS)

У второй и третьей строки дублируется первая часть (это номер карты). Подскажите, каким образом почистить лог от подобных дублей?

если тебе нужны уникальные значения первого ключа, то так:

$ sort -k1 -u tt.txt 

2008000134624 | card exist(52935) | anketa exist(10842) | Sex present | Bday is present | Phone is present | no correct E-mail  | upadte Source (AZS)
2008000286248 | card exist(41574) | anketa exist(10841) | Sex present | Bday is present | Phone is present | no correct E-mail  | Source is present
2010000002810 | card exist(72032) | anketa exist(10843) | Sex present | Bday is present | Phone is present | no correct E-mail  | upadte Source (AZS)
2010000312728 | card exist(142748) | anketa exist(24628) | Sex present | Bday is present | Phone is present | E-mail is present | upadte Source (AZS)
2010000312728 | card exist(72032) | anketa exist(24628) | Sex present | Bday is present | Phone is present | no correct E-mail  | Source is present
emulek ★
()

а если ты желаешь ещё по третьему полю отсортировать, то можно ещё и так:

$ sort -k1 -u tt.txt | sort -t'|' -k3 

2008000286248 | card exist(41574) | anketa exist(10841) | Sex present | Bday is present | Phone is present | no correct E-mail  | Source is present
2008000134624 | card exist(52935) | anketa exist(10842) | Sex present | Bday is present | Phone is present | no correct E-mail  | upadte Source (AZS)
2010000002810 | card exist(72032) | anketa exist(10843) | Sex present | Bday is present | Phone is present | no correct E-mail  | upadte Source (AZS)
2010000312728 | card exist(142748) | anketa exist(24628) | Sex present | Bday is present | Phone is present | E-mail is present | upadte Source (AZS)
2010000312728 | card exist(72032) | anketa exist(24628) | Sex present | Bday is present | Phone is present | no correct E-mail  | Source is present

(это по «anketa»)

emulek ★
()
Вы не можете добавлять комментарии в эту тему. Тема перемещена в архив.
Development