|
Elena Laskavaia
02/27/2009 9:46 AM
post23089
|
I created simple resmgr, only overrode read and write functions.
It manages each resource in separate thread.
The problem is:
I have client code.I cannot control client code. Client creates resource, writes to it, when open for reading,reads,
then unlinks the resource without closing.
When client unlinks the resource resmgr thread
is going down. At this point I have memory leak of allocated ocb memory (because client did not close it).
Client is daemon, so it never exists, so I cannot expect it closing ocb ever. How can I free the memory
when resmgr shuts down? Should I close it (all of them) on unlink? How I can access it (open ocb list)?
|
|
|
|
|
|
David Sarrazin
02/27/2009 10:31 AM
post23102
|
My understanding is this happens:
Client code:
LOOP
OPEN filename
WRITE to filename
READ from filename
UNLINK filename
END LOOP
And everytime the client does the OPEN, your resmgr allocates some
memory to hold the resource?
In this case, client has an fd leak. It is expected that the resmgr
will grow in memory. The resmgr must not release the OCB or the data
connected to the OCB until an IO_CLOSE message is received. When the
client exits, the kernel will send an IO_CLOSE to every open fd, so that
the resmgr can clean up properly.
Have I understood the problem correctly?
David
> -----Original Message-----
> From: Elena Laskavaia [mailto:community-noreply@qnx.com]
> Sent: February 27, 2009 9:46 AM
> To: ostech-core_os
> Subject: resmgr memory leak
>
> I created simple resmgr, only overrode read and write functions.
> It manages each resource in separate thread.
> The problem is:
> I have client code.I cannot control client code. Client
> creates resource, writes to it, when open for reading,reads,
> then unlinks the resource without closing.
> When client unlinks the resource resmgr thread is going down.
> At this point I have memory leak of allocated ocb memory
> (because client did not close it).
> Client is daemon, so it never exists, so I cannot expect it
> closing ocb ever. How can I free the memory when resmgr shuts
> down? Should I close it (all of them) on unlink? How I can
> access it (open ocb list)?
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post23089
>
>
|
|
|
|
|
|
Elena Laskavaia
02/27/2009 10:37 AM
post23103
|
Yes. As I mention client never exists. The problem is resmgr it does not hold ocb, it is memory leak,
so the memory is not referenced anywhere (this memory allocation from iofunc_open_default). I want to close ocb on
unlink. How can I access list of open ocb's?
David Sarrazin wrote:
> My understanding is this happens:
>
> Client code:
>
> LOOP
> OPEN filename
> WRITE to filename
> READ from filename
> UNLINK filename
> END LOOP
>
> And everytime the client does the OPEN, your resmgr allocates some
> memory to hold the resource?
>
> In this case, client has an fd leak. It is expected that the resmgr
> will grow in memory. The resmgr must not release the OCB or the data
> connected to the OCB until an IO_CLOSE message is received. When the
> client exits, the kernel will send an IO_CLOSE to every open fd, so that
> the resmgr can clean up properly.
>
> Have I understood the problem correctly?
>
> David
>
>> -----Original Message-----
>> From: Elena Laskavaia [mailto:community-noreply@qnx.com]
>> Sent: February 27, 2009 9:46 AM
>> To: ostech-core_os
>> Subject: resmgr memory leak
>>
>> I created simple resmgr, only overrode read and write functions.
>> It manages each resource in separate thread.
>> The problem is:
>> I have client code.I cannot control client code. Client
>> creates resource, writes to it, when open for reading,reads,
>> then unlinks the resource without closing.
>> When client unlinks the resource resmgr thread is going down.
>> At this point I have memory leak of allocated ocb memory
>> (because client did not close it).
>> Client is daemon, so it never exists, so I cannot expect it
>> closing ocb ever. How can I free the memory when resmgr shuts
>> down? Should I close it (all of them) on unlink? How I can
>> access it (open ocb list)?
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post23089
>>
>>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post23102
>
|
|
|
|
|
|
Oleh Derevenko(deleted)
02/27/2009 12:13 PM
post23114
|
You can't do anything about it. Client must not unlink resource without prior closing it.
If I have a file open and someone erases that file, I still can access the file via file descriptor I have even though
it's not visible in directory any more. You can't free any resources until clients have them open.
|
|
|
|
|
|
Oleh Derevenko(deleted)
02/27/2009 12:17 PM
post23116
|
To be correct, it does not matter if client first closes resource and then unlinks the file or if it first unlinks the
file and then closes file descriptor it holds. But client MUST close the file.
|
|
|
|
|
|
Elena Laskavaia
02/27/2009 12:29 PM
post23119
|
It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is stopped
and ocb descriptor is not available anymore (leaked).
Oleh Derevenko wrote:
> You can't do anything about it. Client must not unlink resource without prior closing it.
> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even though
it's not visible in directory any more. You can't free any resources until clients have them open.
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post23114
>
|
|
|
|
|
|
Oleh Derevenko(deleted)
02/27/2009 5:41 PM
post23151
|
Can't you create a resource manager at one level up? For example do not create "my_device" in "/dev/" but rather create
"dev" with item "my_device" in it in "/", so that when client unlinks the resource your resource manager remained alive?
|
|
|
|
|
|
Colin Burgess(deleted)
03/05/2009 10:47 AM
post23612
|
You are confusing the notion of unlink.
In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the resource
manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
is no-longer a resmgr listening.
However the memory for the OCBs and the fds are still there, hence the leak.
Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
are asking for the resmgr framework to do this job for you when you destroy it?
Elena Laskavaia wrote:
> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is stopped
and ocb descriptor is not available anymore (leaked).
>
> Oleh Derevenko wrote:
>> You can't do anything about it. Client must not unlink resource without prior closing it.
>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post23114
>>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post23119
>
--
cburgess@qnx.com
|
|
|
|
|
|
Elena Laskavaia
03/05/2009 12:52 PM
post23638
|
Ok, it was not unlink, it was my manager calling dispatch_context_free.
This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
So framework should release all memory it allocated and if was not release by others means.
Colin Burgess wrote:
> You are confusing the notion of unlink.
>
> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the resource
> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
> is no-longer a resmgr listening.
>
> However the memory for the OCBs and the fds are still there, hence the leak.
>
> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
> are asking for the resmgr framework to do this job for you when you destroy it?
>
> Elena Laskavaia wrote:
>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>
>> Oleh Derevenko wrote:
>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>
>>> _______________________________________________
>>> OSTech
>>> http://community.qnx.com/sf/go/post23114
>>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post23119
>>
>
|
|
|
|
|
|
Colin Burgess(deleted)
03/05/2009 1:24 PM
post23650
|
I think you need to call resmgr_detach() before calling destroying your dispatch context.
Elena Laskavaia wrote:
> Ok, it was not unlink, it was my manager calling dispatch_context_free.
> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
> So framework should release all memory it allocated and if was not release by others means.
>
> Colin Burgess wrote:
>> You are confusing the notion of unlink.
>>
>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the resource
>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>> is no-longer a resmgr listening.
>>
>> However the memory for the OCBs and the fds are still there, hence the leak.
>>
>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>> are asking for the resmgr framework to do this job for you when you destroy it?
>>
>> Elena Laskavaia wrote:
>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>
>>> Oleh Derevenko wrote:
>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>
>>>> _______________________________________________
>>>> OSTech
>>>> http://community.qnx.com/sf/go/post23114
>>>>
>>> _______________________________________________
>>> OSTech
>>> http://community.qnx.com/sf/go/post23119
>>>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post23638
>
--
cburgess@qnx.com
|
|
|
|
|
|
Elena Laskavaia
03/05/2009 1:29 PM
post23654
|
I do.
rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
Colin Burgess wrote:
> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>
> Elena Laskavaia wrote:
>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
>> So framework should release all memory it allocated and if was not release by others means.
>>
>> Colin Burgess wrote:
>>> You are confusing the notion of unlink.
>>>
>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the resource
>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>> is no-longer a resmgr listening.
>>>
>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>
>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>
>>> Elena Laskavaia wrote:
>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>>
>>>> Oleh Derevenko wrote:
>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>
>>>>> _______________________________________________
>>>>> OSTech
>>>>> http://community.qnx.com/sf/go/post23114
>>>>>
>>>> _______________________________________________
>>>> OSTech
>>>> http://community.qnx.com/sf/go/post23119
>>>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post23638
>>
>
|
|
|
|
|
|
Colin Burgess(deleted)
03/05/2009 1:35 PM
post23656
|
You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
Elena Laskavaia wrote:
> I do.
> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>
> Colin Burgess wrote:
>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>
>> Elena Laskavaia wrote:
>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
>>> So framework should release all memory it allocated and if was not release by others means.
>>>
>>> Colin Burgess wrote:
>>>> You are confusing the notion of unlink.
>>>>
>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the resource
>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>>> is no-longer a resmgr listening.
>>>>
>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>
>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>
>>>> Elena Laskavaia wrote:
>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>>>
>>>>> Oleh Derevenko wrote:
>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>>
>>>>>> _______________________________________________
>>>>>> OSTech
>>>>>> http://community.qnx.com/sf/go/post23114
>>>>>>
>>>>> _______________________________________________
>>>>> OSTech
>>>>> http://community.qnx.com/sf/go/post23119
>>>>>
>>> _______________________________________________
>>> OSTech
>>> http://community.qnx.com/sf/go/post23638
>>>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post23654
>
--
cburgess@qnx.com
|
|
|
|
|
|
Elena Laskavaia
03/05/2009 1:44 PM
post23659
|
If it works we can just re-direct my pr to docs. I will try it asap.
Colin Burgess wrote:
> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>
> Elena Laskavaia wrote:
>> I do.
>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>
>> Colin Burgess wrote:
>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>
>>> Elena Laskavaia wrote:
>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>
>>>> Colin Burgess wrote:
>>>>> You are confusing the notion of unlink.
>>>>>
>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the resource
>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>>>> is no-longer a resmgr listening.
>>>>>
>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>
>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>
>>>>> Elena Laskavaia wrote:
>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>>>>
>>>>>> Oleh Derevenko wrote:
>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OSTech
>>>>>>> http://community.qnx.com/sf/go/post23114
>>>>>>>
>>>>>> _______________________________________________
>>>>>> OSTech
>>>>>> http://community.qnx.com/sf/go/post23119
>>>>>>
>>>> _______________________________________________
>>>> OSTech
>>>> http://community.qnx.com/sf/go/post23638
>>>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post23654
>>
>
|
|
|
|
|
|
Elena Laskavaia
03/05/2009 1:54 PM
post23663
|
Hmm. Does not seems to have any impact. Exact same memory leaks...
Colin Burgess wrote:
> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>
> Elena Laskavaia wrote:
>> I do.
>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>
>> Colin Burgess wrote:
>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>
>>> Elena Laskavaia wrote:
>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>
>>>> Colin Burgess wrote:
>>>>> You are confusing the notion of unlink.
>>>>>
>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the resource
>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>>>> is no-longer a resmgr listening.
>>>>>
>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>
>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>
>>>>> Elena Laskavaia wrote:
>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>>>>
>>>>>> Oleh Derevenko wrote:
>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OSTech
>>>>>>> http://community.qnx.com/sf/go/post23114
>>>>>>>
>>>>>> _______________________________________________
>>>>>> OSTech
>>>>>> http://community.qnx.com/sf/go/post23119
>>>>>>
>>>> _______________________________________________
>>>> OSTech
>>>> http://community.qnx.com/sf/go/post23638
>>>>
>> _______________________________________________
>> OSTech
>> http://community.qnx.com/sf/go/post23654
>>
>
|
|
|
|
|
|
Colin Burgess(deleted)
03/05/2009 1:55 PM
post23664
|
Are your close handlers being called?
Elena Laskavaia wrote:
> Hmm. Does not seems to have any impact. Exact same memory leaks...
>
>
> Colin Burgess wrote:
>> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>>
>> Elena Laskavaia wrote:
>>> I do.
>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>>
>>> Colin Burgess wrote:
>>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>>
>>>> Elena Laskavaia wrote:
>>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
>>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>>
>>>>> Colin Burgess wrote:
>>>>>> You are confusing the notion of unlink.
>>>>>>
>>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the
resource
>>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>>>>> is no-longer a resmgr listening.
>>>>>>
>>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>>
>>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>>
>>>>>> Elena Laskavaia wrote:
>>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>>>>>
>>>>>>> Oleh Derevenko wrote:
>>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have even
though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> OSTech
>>>>>>>> http://community.qnx.com/sf/go/post23114
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OSTech
>>>>>>> http://community.qnx.com/sf/go/post23119
>>>>>>>
>>>>> _______________________________________________
>>>>> OSTech
>>>>> http://community.qnx.com/sf/go/post23638
>>>>>
>>> _______________________________________________
>>> OSTech
>>> http://community.qnx.com/sf/go/post23654
>>>
>
> _______________________________________________
> OSTech
> http://community.qnx.com/sf/go/post23663
>
--
cburgess@qnx.com
|
|
|
|
|
|
Elena Laskavaia
03/05/2009 2:05 PM
post23667
|
Nope. I put breakpoint at
rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_CLOSE);
and in handle_ocb_close
When I hit resmgr_detach I do step over - breakpoint in handler is not hit.
(base 6.4.0 btw)
(detach runs from different thread than resource manager dispatch)
Colin Burgess wrote:
> Are your close handlers being called?
>
> Elena Laskavaia wrote:
>> Hmm. Does not seems to have any impact. Exact same memory leaks...
>>
>>
>> Colin Burgess wrote:
>>> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>>>
>>> Elena Laskavaia wrote:
>>>> I do.
>>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>>>
>>>> Colin Burgess wrote:
>>>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>>>
>>>>> Elena Laskavaia wrote:
>>>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
>>>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>>>
>>>>>> Colin Burgess wrote:
>>>>>>> You are confusing the notion of unlink.
>>>>>>>
>>>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the
resource
>>>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>>>>>> is no-longer a resmgr listening.
>>>>>>>
>>>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>>>
>>>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>>>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>>>
>>>>>>> Elena Laskavaia wrote:
>>>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>>>>>>
>>>>>>>> Oleh Derevenko wrote:
>>>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have
even though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> OSTech
>>>>>>>>> http://community.qnx.com/sf/go/post23114
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> OSTech
>>>>>>>> http://community.qnx.com/sf/go/post23119
>>>>>>>>
>>>>>> _______________________________________________
>>>>>> OSTech
>>>>>> http://community.qnx.com/sf/go/post23638
>>>>>>
>>>> _______________________________________________
>>>> OSTech
>>>>...
|
|
|
|
|
|
Colin Burgess(deleted)
03/05/2009 3:35 PM
post23705
|
Hmmm, resmgr_detach calls _resmgr_detach_id() which is supposed to call your io_close and io_close_ocb handlers...
I wonder what's going on?
Elena Laskavaia wrote:
> Nope. I put breakpoint at
> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_CLOSE);
>
> and in handle_ocb_close
>
> When I hit resmgr_detach I do step over - breakpoint in handler is not hit.
> (base 6.4.0 btw)
>
> (detach runs from different thread than resource manager dispatch)
>
>
> Colin Burgess wrote:
>> Are your close handlers being called?
>>
>> Elena Laskavaia wrote:
>>> Hmm. Does not seems to have any impact. Exact same memory leaks...
>>>
>>>
>>> Colin Burgess wrote:
>>>> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>>>>
>>>> Elena Laskavaia wrote:
>>>>> I do.
>>>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>>>>
>>>>> Colin Burgess wrote:
>>>>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>>>>
>>>>>> Elena Laskavaia wrote:
>>>>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call "
close_all_ocbs".
>>>>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>>>>
>>>>>>> Colin Burgess wrote:
>>>>>>>> You are confusing the notion of unlink.
>>>>>>>>
>>>>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the
resource
>>>>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>>>>>>> is no-longer a resmgr listening.
>>>>>>>>
>>>>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>>>>
>>>>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess you
>>>>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>>>>
>>>>>>>> Elena Laskavaia wrote:
>>>>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch is
stopped and ocb descriptor is not available anymore (leaked).
>>>>>>>>>
>>>>>>>>> Oleh Derevenko wrote:
>>>>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have
even though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> OSTech
>>>>>>>>>> http://community.qnx.com/sf/go/post23114
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> OSTech
>>>>>>>>>...
|
|
|
|
|
|
Elena Laskavaia
03/05/2009 3:47 PM
post23710
|
If you send me debug libc (or where the code is?) I can debug it.
Colin Burgess wrote:
> Hmmm, resmgr_detach calls _resmgr_detach_id() which is supposed to call your io_close and io_close_ocb handlers...
>
> I wonder what's going on?
>
> Elena Laskavaia wrote:
>> Nope. I put breakpoint at
>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_CLOSE);
>>
>> and in handle_ocb_close
>>
>> When I hit resmgr_detach I do step over - breakpoint in handler is not hit.
>> (base 6.4.0 btw)
>>
>> (detach runs from different thread than resource manager dispatch)
>>
>>
>> Colin Burgess wrote:
>>> Are your close handlers being called?
>>>
>>> Elena Laskavaia wrote:
>>>> Hmm. Does not seems to have any impact. Exact same memory leaks...
>>>>
>>>>
>>>> Colin Burgess wrote:
>>>>> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>>>>>
>>>>> Elena Laskavaia wrote:
>>>>>> I do.
>>>>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>>>>>
>>>>>> Colin Burgess wrote:
>>>>>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>>>>>
>>>>>>> Elena Laskavaia wrote:
>>>>>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>>>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call
"close_all_ocbs".
>>>>>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>>>>>
>>>>>>>> Colin Burgess wrote:
>>>>>>>>> You are confusing the notion of unlink.
>>>>>>>>>
>>>>>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the
resource
>>>>>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because there
>>>>>>>>> is no-longer a resmgr listening.
>>>>>>>>>
>>>>>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>>>>>
>>>>>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess
you
>>>>>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>>>>>
>>>>>>>>> Elena Laskavaia wrote:
>>>>>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch
is stopped and ocb descriptor is not available anymore (leaked).
>>>>>>>>>>
>>>>>>>>>> Oleh Derevenko wrote:
>>>>>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>>>>>> If I have a file open and someone erases that file, I still can access the file via file descriptor I have
even though it's not visible in directory any more. You can't free any resources until clients have them open.
>>>>>>>>>>>
>>>>>>>>>>>...
|
|
|
|
|
|
Colin Burgess(deleted)
03/05/2009 3:53 PM
post23711
|
//athena/cburgess$/elena/libc.so.3
Elena Laskavaia wrote:
> If you send me debug libc (or where the code is?) I can debug it.
>
> Colin Burgess wrote:
>> Hmmm, resmgr_detach calls _resmgr_detach_id() which is supposed to call your io_close and io_close_ocb handlers...
>>
>> I wonder what's going on?
>>
>> Elena Laskavaia wrote:
>>> Nope. I put breakpoint at
>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_CLOSE);
>>>
>>> and in handle_ocb_close
>>>
>>> When I hit resmgr_detach I do step over - breakpoint in handler is not hit.
>>> (base 6.4.0 btw)
>>>
>>> (detach runs from different thread than resource manager dispatch)
>>>
>>>
>>> Colin Burgess wrote:
>>>> Are your close handlers being called?
>>>>
>>>> Elena Laskavaia wrote:
>>>>> Hmm. Does not seems to have any impact. Exact same memory leaks...
>>>>>
>>>>>
>>>>> Colin Burgess wrote:
>>>>>> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>>>>>>
>>>>>> Elena Laskavaia wrote:
>>>>>>> I do.
>>>>>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>>>>>>
>>>>>>> Colin Burgess wrote:
>>>>>>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>>>>>>
>>>>>>>> Elena Laskavaia wrote:
>>>>>>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>>>>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call
"close_all_ocbs".
>>>>>>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>>>>>>
>>>>>>>>> Colin Burgess wrote:
>>>>>>>>>> You are confusing the notion of unlink.
>>>>>>>>>>
>>>>>>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the
resource
>>>>>>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because
there
>>>>>>>>>> is no-longer a resmgr listening.
>>>>>>>>>>
>>>>>>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>>>>>>
>>>>>>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess
you
>>>>>>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>>>>>>
>>>>>>>>>> Elena Laskavaia wrote:
>>>>>>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch
is stopped and ocb descriptor is not available anymore (leaked).
>>>>>>>>>>>
>>>>>>>>>>> Oleh Derevenko wrote:
>>>>>>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>>>>>>> If I have a file open and someone erases that file, I still can access the file via...
|
|
|
|
|
|
Colin Burgess(deleted)
03/06/2009 11:06 AM
post23783
|
Did you get anywhere with it?
Elena Laskavaia wrote:
> If you send me debug libc (or where the code is?) I can debug it.
>
> Colin Burgess wrote:
>> Hmmm, resmgr_detach calls _resmgr_detach_id() which is supposed to call your io_close and io_close_ocb handlers...
>>
>> I wonder what's going on?
>>
>> Elena Laskavaia wrote:
>>> Nope. I put breakpoint at
>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_CLOSE);
>>>
>>> and in handle_ocb_close
>>>
>>> When I hit resmgr_detach I do step over - breakpoint in handler is not hit.
>>> (base 6.4.0 btw)
>>>
>>> (detach runs from different thread than resource manager dispatch)
>>>
>>>
>>> Colin Burgess wrote:
>>>> Are your close handlers being called?
>>>>
>>>> Elena Laskavaia wrote:
>>>>> Hmm. Does not seems to have any impact. Exact same memory leaks...
>>>>>
>>>>>
>>>>> Colin Burgess wrote:
>>>>>> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>>>>>>
>>>>>> Elena Laskavaia wrote:
>>>>>>> I do.
>>>>>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>>>>>>
>>>>>>> Colin Burgess wrote:
>>>>>>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>>>>>>
>>>>>>>> Elena Laskavaia wrote:
>>>>>>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>>>>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to call
"close_all_ocbs".
>>>>>>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>>>>>>
>>>>>>>>> Colin Burgess wrote:
>>>>>>>>>> You are confusing the notion of unlink.
>>>>>>>>>>
>>>>>>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the
resource
>>>>>>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because
there
>>>>>>>>>> is no-longer a resmgr listening.
>>>>>>>>>>
>>>>>>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>>>>>>
>>>>>>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess
you
>>>>>>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>>>>>>
>>>>>>>>>> Elena Laskavaia wrote:
>>>>>>>>>>> It is not the case. In my case client cannot read from resource after it has been unlinked because dispatch
is stopped and ocb descriptor is not available anymore (leaked).
>>>>>>>>>>>
>>>>>>>>>>> Oleh Derevenko wrote:
>>>>>>>>>>>> You can't do anything about it. Client must not unlink resource without prior closing it.
>>>>>>>>>>>> If I have a file open and someone erases that file, I still can access the file via file...
|
|
|
|
|
|
Elena Laskavaia
03/06/2009 11:42 AM
post23795
|
Not yet. I took trunk libc and it seems like working on simple case (with this flag is does call ocb_close).
But when I run automated test case it still has leaks,
I am working on trying to pin it down. Could it be race condition of some sort? resmgr_detach and dispatch_destroy are
called from main thread
and dispatch_context_free from another.
Colin Burgess wrote:
> Did you get anywhere with it?
>
> Elena Laskavaia wrote:
>> If you send me debug libc (or where the code is?) I can debug it.
>>
>> Colin Burgess wrote:
>>> Hmmm, resmgr_detach calls _resmgr_detach_id() which is supposed to call your io_close and io_close_ocb handlers...
>>>
>>> I wonder what's going on?
>>>
>>> Elena Laskavaia wrote:
>>>> Nope. I put breakpoint at
>>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_CLOSE);
>>>>
>>>> and in handle_ocb_close
>>>>
>>>> When I hit resmgr_detach I do step over - breakpoint in handler is not hit.
>>>> (base 6.4.0 btw)
>>>>
>>>> (detach runs from different thread than resource manager dispatch)
>>>>
>>>>
>>>> Colin Burgess wrote:
>>>>> Are your close handlers being called?
>>>>>
>>>>> Elena Laskavaia wrote:
>>>>>> Hmm. Does not seems to have any impact. Exact same memory leaks...
>>>>>>
>>>>>>
>>>>>> Colin Burgess wrote:
>>>>>>> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this isn't documented...
>>>>>>>
>>>>>>> Elena Laskavaia wrote:
>>>>>>>> I do.
>>>>>>>> rc=resmgr_detach(elem->dpp,elem->attach_id,_RESMGR_DETACH_ALL);
>>>>>>>>
>>>>>>>> Colin Burgess wrote:
>>>>>>>>> I think you need to call resmgr_detach() before calling destroying your dispatch context.
>>>>>>>>>
>>>>>>>>> Elena Laskavaia wrote:
>>>>>>>>>> Ok, it was not unlink, it was my manager calling dispatch_context_free.
>>>>>>>>>> This call creates memory leaks. Framework does not provide an API to access ocb's, and there is no way to
call "close_all_ocbs".
>>>>>>>>>> So framework should release all memory it allocated and if was not release by others means.
>>>>>>>>>>
>>>>>>>>>> Colin Burgess wrote:
>>>>>>>>>>> You are confusing the notion of unlink.
>>>>>>>>>>>
>>>>>>>>>>> In the PR you submitted, you show that you are not calling unlink, rather you are sending a SIGNAL to the
resource
>>>>>>>>>>> manager, which is then shutting down the resmgr. That is why the client cannot close() the fds, because
there
>>>>>>>>>>> is no-longer a resmgr listening.
>>>>>>>>>>>
>>>>>>>>>>> However the memory for the OCBs and the fds are still there, hence the leak.
>>>>>>>>>>>
>>>>>>>>>>> Right now, it is up to you in this signal handling code to initiate cleanup of the remaining ocbs. I guess
you
>>>>>>>>>>> are asking for the resmgr framework to do this job for you when you destroy it?
>>>>>>>>>>>
>>>>>>>>>>> Elena Laskavaia...
View Full Message
|
|
|
|
|
|
Steve Reid
03/06/2009 10:57 AM
post23781
|
> You need to also add _RESMGR_DETACH_CLOSE to the flags. For some reason this
> isn't documented...
I've created PR 66106 to address this.
|
|
|
|
|
|
Xiaodan Tang(deleted)
02/27/2009 11:06 PM
post23155
|
Well, I wouldn't say "you can't do it".
A standard resource manager, is more or less based on a POSIX file system behavior. The standard POSIX file system, is
that you can unlink the file, and hold and open fd ...
However, if this is your own resource manager, and you want to DEFINE that "unlink the resource, means all open fd will
be invalide" is your resource manager's behavior, I don't see why you can't.
You can certainly in your unlink call out, find all the associated ocbs, "close()" them as you wish, clean up the
resource. You won't have any memory leak in your resmgr.
The client however, probably would have a fd leak, if they try to access the fd after an unlink(), they will receive an
error.
|
|
|
|
|
|
Oleh Derevenko(deleted)
02/28/2009 4:49 AM
post23158
|
> Well, I wouldn't say "you can't do it".
>
> However, if this is your own resource manager, and you want to DEFINE that "
> unlink the resource, means all open fd will be invalide" is your resource
> manager's behavior, I don't see why you can't.
>
> You can certainly in your unlink call out, find all the associated ocbs, "
> close()" them as you wish, clean up the resource. You won't have any memory
> leak in your resmgr.
>
> The client however, probably would have a fd leak, if they try to access the
> fd after an unlink(), they will receive an error.
Yes, indeed. I use this technique for myself too. If you need to terminate a file, you just set a flag in OCB to mark it
as terminated, free all the associated resources and reject all the subsequent requests for OCB except for close().
This way you can free majority of resources and prevent access to internal objects in server that need to be deleted.
However, some memory still remains leaked because you can't free the OCB itself.
|
|
|
|
|
|
