Aesthetic Image Captioning from Weakly-Labelled Photographs
File Type:
PDFItem Type:
Conference PaperDate:
2019Access:
openAccessCitation:
K. Ghosal, A. Rana and A. Smolic, "Aesthetic Image Captioning From Weakly-Labelled Photographs," 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019, pp. 4550-4560Download Item:
Abstract:
Aesthetic image captioning (AIC) refers to the multimodal
task of generating critical textual feedbacks for photographs.
While in natural image captioning (NIC), deep
models are trained in an end-to-end manner using large
curated datasets such as MS-COCO, no such large-scale,
clean dataset exists for AIC. Towards this goal, we propose
an automatic cleaning strategy to create a benchmarking
AIC dataset, by exploiting the images and noisy comments
easily available from photography websites. We propose a
probabilistic caption-filtering method for cleaning the noisy
web-data, and compile a large-scale, clean dataset ‘AVACaptions’,
( ∼ 230, 000 images with ∼ 5 captions per image).
Additionally, by exploiting the latent associations between
aesthetic attributes, we propose a strategy for training
a convolutional neural network (CNN) based visual feature
extractor, typically the first component of an AIC framework.
The strategy is weakly supervised and can be effectively
used to learn rich aesthetic representations, without
requiring expensive ground-truth annotations. We finally
showcase a thorough analysis of the proposed contributions
using automatic metrics and subjective evaluations.
URI:
https://v-sense.scss.tcd.ie/wp-content/uploads/2019/08/ICCVW_CROMOL_2019.pdfhttp://hdl.handle.net/2262/91579
Sponsor
Grant Number
SFI stipend
15/RP/2776
Author's Homepage:
http://people.tcd.ie/smolicaDescription:
PUBLISHEDType of material:
Conference PaperURI:
https://v-sense.scss.tcd.ie/wp-content/uploads/2019/08/ICCVW_CROMOL_2019.pdfhttp://hdl.handle.net/2262/91579
Collections:
Availability:
Full text availableSubject (TCD):
Creative Technologies , Multimedia & CreativityDOI:
10.1109/ICCVW.2019.00556Licences: