I have an entity called "Degree": enter image description here

I have a source that retrieves 92 items successfully. enter image description here

Inside of sources configuration, I have the following settings:

Tags definition enter image description here

Document Extractor enter image description here

Document Extractor code

function extract(request, response) {
    $ = response.body;
    url = request.url;
    id = url.replace(/[/:.]/g, '_');

    return [{
        'id': id,
        'title': $('title').text(),
        'name': $('meta[name="og:title"]').attr('content') || 'Title not set',
        'description': $('meta[name="og:description"]').attr('content') || 'Description not set',
        'type': $('meta[name="og:type"]').attr('content') || 'website_content',
        'tags': $('meta[name="tags"]').attr('content') || 'Tags not set',
        'footer_tags': $('meta[name="footer_tags"]').attr('content') || 'Footer Tags not set',

Like I said earlier, 92 items indexed successfully. When I am trying to see indexed content under Content Collection > Degree, there is no content indexed. enter image description here

I don't know why, did I miss something?

Thank you.

2 Answers 2


From what you have posted, I cant see anything that is mapping the incoming 'indexed' items to your 'degree' entity.

So the crawler is successfully finding 92 items, but not mapping any successfully.

If you look to the far right of "degree card" on the image you have labelled "tags definition", you should see an option to edit.

enter image description here

If you click that, it should open up an editer where you define the JS Source code for extracting the content.

You need to then add something like this:

// Sample extractor function. Change the function to suit your individual needs
function extract(request, response) {
    $ = response.body;

    return [{
        'description': $('meta[name="description"]').attr('content') || $('meta[property="og:description"]').attr('content') || $('p').text(),
        'name': $('meta[name="title"]').attr('content'),
        'type': 'video',
        'url': $('meta[property="og:url"]').attr('content'),
        'image': $('meta[property="og:image"]').attr('content')

Making sure to map items that are guaranteed to be on every page being indexed. Otherwise you need to provide a default.

Please try that out and let me know.

  • Hi, I updated the answer with document extractor code.
    – Godiez
    Commented Apr 19 at 11:16

It works now, after a few tests, "Tags" under "Tags definition" should be "content".

  • By changing that to content, you are not indexing and mapping to the 'degree' entity which seems to be the focus of your question. Commented Apr 19 at 21:57
  • Yes, if you look at Document Extractor, screenshot above, entity points to "degree"
    – Godiez
    Commented Apr 22 at 11:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.