A question about the attributes in OpenIE_process.py

[elevenofji]

Hello! I'm a little confused about attributes in .json file, which is mentioned in OpenIE_process.py.

Take a .story in cnn/dm dataset for example:

In process_one function:
filename = os.path.join(DATA_DIR, split, str(_id) + '.json')
How can I get this json file？

Especially， in prepare_data function:

coref = data['**input_mention_cluster**']
abstract = data['**abstract'**]
article = data['**article**']
if split != 'test': extracts = data['**extracted_combine**']

and:

for entity in coref:
entities = []
for mention in entity:
entities.append((mention['**text**'], mention['**position**']))
How can i let a story file turn into a json file like this?

I've tried Stanford corenlp openie like Readme.md in ./preprocess. But here is what I get:

/raw_stories_src/ad021a7924ee57491cee8576499ac196b1afeb23.src 2 worried customers as lines is one 23 25 22 2 13 14 1.000 long lines formed before counters opened at the northern rock building society , one of the uk 's top five lenders , as worried customers ignored reassurances from the bank and the government . RB NNS VBN IN NNS VBD IN DT JJ NN NN NN , CD IN DT NN POS JJ CD NNS , IN JJ NNS VBN NNS IN DT NN CC DT NN . worried customer as line be one

it looks like the triples has been extracted, but I want to turn it into the json file like yours. Could you give me a explanation about how you get these attributes, and what these attributes(input_mention_cluster,extracted_combine....) mean?

Thank you very much !

[elevenofji]

And another quesition： what's the difference between extracted and extracted_combine in your preprocessed data?

[luyang-huang96]

"extracted" and "extracted_combine" are the different strategy to construct extractive summarization label. "input_mention_cluster" is coref results from stanfordcorenlp.

[elevenofji]

there may be another problem:
In raw 334 & 335 of ./models/copy_summ_multiencoder.py :
torch.cat(h.chunk(2, dim=0), dim=2),
torch.cat(c.chunk(2, dim=0), dim=2)
However , the h & c of nn.LSTM have the [num_layersnumdirections , batch_size , dim],so when you make the "chunk" operation (setting the "n_layer" >= 2 ), it will not split the vector into different directions. Right?

[KALLAZ]

Hello! I'm a little confused about attributes in .json file, which is mentioned in OpenIE_process.py.

Take a .story in cnn/dm dataset for example:

In process_one function:
filename = os.path.join(DATA_DIR, split, str(_id) + '.json')
How can I get this json file？

Especially， in prepare_data function:

coref = data['**input_mention_cluster**']
abstract = data['**abstract'**]
article = data['**article**']
if split != 'test': extracts = data['**extracted_combine**']

and:

for entity in coref:
entities = []
for mention in entity:
entities.append((mention['**text**'], mention['**position**']))
How can i let a story file turn into a json file like this?

I've tried Stanford corenlp openie like Readme.md in ./preprocess. But here is what I get:

/raw_stories_src/ad021a7924ee57491cee8576499ac196b1afeb23.src 2 worried customers as lines is one 23 25 22 2 13 14 1.000 long lines formed before counters opened at the northern rock building society , one of the uk 's top five lenders , as worried customers ignored reassurances from the bank and the government . RB NNS VBN IN NNS VBD IN DT JJ NN NN NN , CD IN DT NN POS JJ CD NNS , IN JJ NNS VBN NNS IN DT NN CC DT NN . worried customer as line be one

it looks like the triples has been extracted, but I want to turn it into the json file like yours. Could you give me a explanation about how you get these attributes, and what these attributes(input_mention_cluster,extracted_combine....) mean?

Thank you very much !

Did you solve this problem? I have the same problem as you. @luyang-huang96

[isaacisaactang]

@luyang-huang96 How did you convert cnn/dm dataset to preprocessed data?

[vicozee97]

Did you solve this problem? I have the same problem as you.

Hello, I have the same problem as you also. Did you solve this problem?😊

[pac-man21]

Hello! I'm a little confused about attributes in .json file, which is mentioned in OpenIE_process.py.

Take a .story in cnn/dm dataset for example:

In process_one function: filename = os.path.join(DATA_DIR, split, str(_id) + '.json') How can I get this json file？

Especially， in prepare_data function:

coref = data['**input_mention_cluster**'] abstract = data['**abstract'**] article = data['**article**'] if split != 'test': extracts = data['**extracted_combine**']

and:

for entity in coref: entities = [] for mention in entity: entities.append((mention['**text**'], mention['**position**'])) How can i let a story file turn into a json file like this?

I've tried Stanford corenlp openie like Readme.md in ./preprocess. But here is what I get:

/raw_stories_src/ad021a7924ee57491cee8576499ac196b1afeb23.src 2 worried customers as lines is one 23 25 22 2 13 14 1.000 long lines formed before counters opened at the northern rock building society , one of the uk 's top five lenders , as worried customers ignored reassurances from the bank and the government . RB NNS VBN IN NNS VBD IN DT JJ NN NN NN , CD IN DT NN POS JJ CD NNS , IN JJ NNS VBN NNS IN DT NN CC DT NN . worried customer as line be one

it looks like the triples has been extracted, but I want to turn it into the json file like yours. Could you give me a explanation about how you get these attributes, and what these attributes(input_mention_cluster,extracted_combine....) mean?

Thank you very much !

Hi! Were you able to figure out how to convert the triples into the json file?
@luyang-huang96 can you please elaborate how to go forward with the conversion?

[ly111111111111]

Hello! I'm a little confused about attributes in .json file, which is mentioned in OpenIE_process.py.

Take a .story in cnn/dm dataset for example:

In process_one function: filename = os.path.join(DATA_DIR, split, str(_id) + '.json') How can I get this json file？

Especially， in prepare_data function:

coref = data['**input_mention_cluster**'] abstract = data['**abstract'**] article = data['**article**'] if split != 'test': extracts = data['**extracted_combine**']

and:

for entity in coref: entities = [] for mention in entity: entities.append((mention['**text**'], mention['**position**'])) How can i let a story file turn into a json file like this?

I've tried Stanford corenlp openie like Readme.md in ./preprocess. But here is what I get:

/raw_stories_src/ad021a7924ee57491cee8576499ac196b1afeb23.src 2 worried customers as lines is one 23 25 22 2 13 14 1.000 long lines formed before counters opened at the northern rock building society , one of the uk 's top five lenders , as worried customers ignored reassurances from the bank and the government . RB NNS VBN IN NNS VBD IN DT JJ NN NN NN , CD IN DT NN POS JJ CD NNS , IN JJ NNS VBN NNS IN DT NN CC DT NN . worried customer as line be one

it looks like the triples has been extracted, but I want to turn it into the json file like yours. Could you give me a explanation about how you get these attributes, and what these attributes(input_mention_cluster,extracted_combine....) mean?

Thank you very much !

were you able to figure out how to convert the triples into the json file？ thank u

[ly111111111111]

@elevenofji Have you solved the data problem in Open_IE.py? If so, could you help me？