Skip to content
GitLab
Commit 1a0249c4 authored by Паршина Екатерина Сергеевна's avatar Паршина Екатерина Сергеевна
Browse files

Upload New File

parent 62886134
master
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 398 additions and 0 deletions
+398 -0
HW 4/Task3/task_3.ipynb 0 → 100644
+ 398
− 0
View file @ 1a0249c4
%% Cell type:markdown id:7b42cfcb-38f0-425b-abc0-e7c27fe9e3af tags:
#Задание 3. Часть 2
%% Cell type:markdown id:f53c99ca-93cd-4324-a643-69c0cbbb670d tags:
Необходимо собрать собственный набор данных из **различных** изображений Вашего лица с разных ракурсов, желательно настоящие фотографии из личного архива (20 штук)\
Возьмите эталонное изображение (как в паспорте) и при помощи библиотеки [DeepFace](https://github.com/serengil/deepface) проверьте его на соответствие всему датасету. Посчитайте метрику Precision. \
\
Примените каждую из перечисленных ниже аугментаций (**по-отдельности**) ко всему датасету и измерьте метрику Precision для измененнного датасета:
* Поворот изображения на 45° и 90°.
* Добавление шума (Gaussian Noise).
* Изменение яркости (увеличение и уменьшение на 50%).
* Размытие с различными параметрами.
\
Реузультаты соберите в таблицу вида:
Метод | Исходный датасет | Поворот на 45° | Поворот на 90° | Изображение с шумом |
--- | ----|--- | --- | --- |
VGG-Face | 0 | 0 | 0 | 0 |
Facenet | 0 | 0 | 0 | 0 |
Facenet512 | 0 | 0 | 0 | 0 |
OpenFace | 0 | 0 | 0 | 0 |
DeepFace | 0 | 0 | 0 | 0 |
DeepID | 0 | 0 | 0 | 0 |
ArcFace | 0 | 0 | 0 | 0 |
Dlib | 0 | 0 | 0 | 0 |
SFace | 0 | 0 | 0 | 0 |
GhostFaceNet | 0 | 0 | 0 | 0 |
%% Cell type:code id:6d82b3d9-2e3c-4726-9bdb-72d340c29e5a tags:
``` python
import os
import cv2
import numpy as np
import pandas as pd
from deepface import DeepFace
from sklearn.metrics import recall_score
image_dir = "kate"
reference_image_path = "user.jpg"
models = ['VGG-Face', 'Facenet512', 'OpenFace', 'DeepID', 'ArcFace', 'Dlib', 'SFace', 'GhostFaceNet']
def rotate_image(image, angle):
center = tuple(np.array(image.shape[1::-1]) / 2)
rot_mat = cv2.getRotationMatrix2D(center, angle, 1.0)
return cv2.warpAffine(image, rot_mat, image.shape[1::-1], flags=cv2.INTER_LINEAR)
def add_noise(image):
noise = np.random.normal(loc=0, scale=25, size=image.shape)
noisy_image = cv2.add(image, noise.astype(np.uint8))
return np.clip(noisy_image, 0, 255)
def change_brightness(image, factor):
return cv2.convertScaleAbs(image, alpha=factor, beta=0)
def blur_image(image, kernel_size):
return cv2.GaussianBlur(image, (kernel_size, kernel_size), 0)
def evaluate_images(reference_image, images, model_name):
matches = []
for i, img in enumerate(images):
try:
result = DeepFace.verify(reference_image, img, model_name=model_name, enforce_detection=False)
matches.append(result["verified"])
except Exception as e:
print(f"Ошибка с изображением номер {i}: {e}")
matches.append(False)
return matches
reference_image = cv2.imread(reference_image_path)
if reference_image is None:
raise ValueError(f"Не удалось загрузить эталонное изображение: {reference_image_path}")
images = []
for img_name in os.listdir(image_dir):
if img_name.endswith('.JPG') or img_name.endswith('.jpg'):
img_path = os.path.join(image_dir, img_name)
img = cv2.imread(img_path)
if img is not None:
images.append(img)
else:
print(f"Не удалось загрузить изображение: {img_path}")
if not images:
raise ValueError("Не удалось загрузить изображения из директории.")
results = []
for model in models:
original_matches = evaluate_images(reference_image, images, model)
original_recall = recall_score([1]*len(original_matches), original_matches)
rotated_45 = [rotate_image(img, 45) for img in images]
recall_45 = recall_score([1]*len(rotated_45), evaluate_images(reference_image, rotated_45, model))
rotated_90 = [rotate_image(img, 90) for img in images]
recall_90 = recall_score([1]*len(rotated_90), evaluate_images(reference_image, rotated_90, model))
noisy_images = [add_noise(img) for img in images]
recall_noise = recall_score([1]*len(noisy_images), evaluate_images(reference_image, noisy_images, model))
bright_images_up = [change_brightness(img, 1.5) for img in images]
recall_bright_up = recall_score([1]*len(bright_images_up), evaluate_images(reference_image, bright_images_up, model))
bright_images_down = [change_brightness(img, 0.5) for img in images]
recall_bright_down = recall_score([1]*len(bright_images_down), evaluate_images(reference_image, bright_images_down, model))
blurred_images = [blur_image(img, 5) for img in images]
recall_blur = recall_score([1]*len(blurred_images), evaluate_images(reference_image, blurred_images, model))
results.append([
model,
original_recall,
recall_45,
recall_90,
recall_noise,
recall_bright_up,
recall_bright_down,
recall_blur
])
df_results = pd.DataFrame(
results,
columns=[
'Метод',
'Исходный набор',
'Поворот на 45°',
'Поворот на 90°',
'Изображение с шумом',
'Увеличенная яркость',
'Уменьшенная яркость',
'Размытие'
]
)
print(df_results)
```
%% Output
2025-02-01 14:08:36.104207: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-02-01 14:08:36.113857: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1738408116.124793 227811 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1738408116.128118 227811 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-02-01 14:08:36.139691: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
W0000 00:00:1738408118.299641 227811 gpu_device.cc:2344] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
Метод Исходный набор Поворот на 45° Поворот на 90° \
0 VGG-Face 0.60 0.05 0.00
1 Facenet512 0.20 0.00 0.00
2 OpenFace 0.00 0.00 0.00
3 DeepID 0.15 0.05 0.05
4 ArcFace 0.55 0.05 0.00
5 Dlib 0.60 0.05 0.00
6 SFace 0.45 0.05 0.00
7 GhostFaceNet 0.50 0.05 0.00
Изображение с шумом Увеличенная яркость Уменьшенная яркость Размытие
0 0.05 0.55 0.65 0.60
1 0.00 0.35 0.15 0.25
2 0.00 0.00 0.00 0.05
3 0.00 0.15 0.15 0.15
4 0.00 0.55 0.65 0.60
5 0.10 0.50 0.70 0.65
6 0.00 0.45 0.60 0.50
7 0.05 0.40 0.55 0.55
%% Cell type:markdown id:4d4e15a8-aa3c-454a-8aba-c65a29bda5ec tags:
#Задание 3. Часть 2
%% Cell type:markdown id:1e002360-3c96-4393-8cc1-e5e68bedcb8b tags:
В данном задании предлагается рассмотреть задачу распознавания лиц как задачу классификации для оценки точности.
Вам даны [видео](https://drive.google.com/drive/folders/1z_YCdJF2Rf9WzlNSY3BpNFiakNisq5VB?usp=sharing), для которых представлена разметка в виде тайм-кодов и база фотографий людей с этих видео.
Необходимо взять каждый 50-й кадр видео (способ разбиения на кадры с учётом разметки - на ваше усмотрение) и для полученного набора изображений оценить метрику Precision на данном наборе изображений для всех лиц, присутствующих на видео и в разметке.
%% Cell type:code id:3b20127c-94a4-41b9-9971-66f92346ca32 tags:
``` python
import os
import cv2
import pandas as pd
from deepface import DeepFace
def time_to_frame(time_str, fps):
"""Преобразует строку времени формата 'HH:MM:SS' в номер кадра."""
time_parts = time_str.split(':')
if len(time_parts) == 3:
hours, minutes, seconds = map(float, time_parts)
else:
raise ValueError("Формат времени должен быть 'HH:MM:SS'")
total_seconds = hours * 3600 + minutes * 60 + seconds
return int(total_seconds * fps)
def get_labeled_persons_for_frame(labels, frame_number):
"""Возвращает список лиц, присутствующих на данном фрейме, исходя из разметки."""
for start, end, persons in labels:
if start <= frame_number <= end:
return persons
return []
def extract_labels(df, fps):
"""Извлекает и преобразует метки из CSV-файла в формат, пригодный для поиска лиц."""
labels = [
(time_to_frame(row['from'], fps), time_to_frame(row['to'], fps), row['persons'].replace(',', ' ').split())
for _, row in df.iterrows()
]
return labels
def match_verified(frame, person_photos_path):
"""Проверяет, есть ли в текущем кадре изображение указанного человека."""
if not os.path.exists(person_photos_path):
return False
for filename in os.listdir(person_photos_path):
image_path = os.path.join(person_photos_path, filename)
try:
result = DeepFace.verify(image_path, frame, enforce_detection=False)
if result['verified']:
return True
except Exception:
continue
return False
def evaluate_frame(frame, frame_number, labels, photos_dir):
"""Оценивает текущий кадр для всех меток и возвращает количество верных и неверных сопоставлений."""
labeled_persons = get_labeled_persons_for_frame(labels, frame_number)
tp, fn = 0, 0
for person in labeled_persons:
person_photos_path = os.path.join(photos_dir, person)
if match_verified(frame, person_photos_path):
tp += 1
else:
fn += 1
return tp, fn
def process_video(video_path, labels_path, photos_dir):
"""Обрабатывает видео, возвращая количество истинных и ложных совпадений."""
cap = cv2.VideoCapture(video_path)
df = pd.read_csv(labels_path)
fps = cap.get(cv2.CAP_PROP_FPS)
labels = extract_labels(df, fps)
tp_total, fn_total = 0, 0
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
frame_number = 0
while frame_number < total_frames:
ret, frame = cap.read()
if not ret:
break
tp, fn = evaluate_frame(frame, frame_number, labels, photos_dir)
tp_total += tp
fn_total += fn
frame_number += 50
cap.set(cv2.CAP_PROP_POS_FRAMES, frame_number)
cap.release()
return tp_total, fn_total
def calculate_recall(tp, fn):
return tp / (tp + fn) if (tp + fn) > 0 else 0
def main(videos_dir, labels_dir, photos_dir):
overall_recalls = []
for video_filename in os.listdir(videos_dir):
if not video_filename.lower().endswith('.mp4'):
continue
video_path = os.path.join(videos_dir, video_filename)
labels_path = os.path.join(labels_dir, os.path.splitext(video_filename)[0] + ".csv")
tp, fn = process_video(video_path, labels_path, photos_dir)
recall = calculate_recall(tp, fn)
overall_recalls.append(recall)
print(f"Видео: {video_filename}, Recall: {recall:.4f}")
if overall_recalls:
average_recall = sum(overall_recalls) / len(overall_recalls)
print(f"Средний Recall для всех видео: {average_recall:.4f}")
videos_dir = "task3/videos"
labels_dir = "task3/labels"
photos_dir = "task3/photos"
main(videos_dir, labels_dir, photos_dir)
```
%% Output
2025-02-01 09:08:30.181388: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-02-01 09:08:30.191264: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1738390110.202245 179413 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1738390110.205594 179413 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-02-01 09:08:30.217277: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
W0000 00:00:1738390111.407309 179413 gpu_device.cc:2344] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
Видео: palmasizade_video.mp4, Recall: 0.1489
Средний Recall для всех видео: 0.1489
%% Cell type:code id:3d113f6f-697c-4e69-9680-88e2b568c68a tags:
``` python
```
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment