hi,
It's a little emergency.
I have a window form project that the window form embeded a web browser object-AxSHDocVw.
When the program finish navigation the URL and show the web page in the web browser object, the program go to parse the DOM tree.
I write the parsing program into the docuemtcomplete handle. Sometime the system trigger the documentcomplete handle, but the web browser was not really download complete the whole html document.
It will cause the DOM tree parsing error.
How can I avoid this situation
And let me ensure that the navigation is really finished completely!
Thanks a lot!
Y. J.
Sample program:
Private Sub wb_DocumentComplete(ByVal sender As System.Object, ByVal e As AxSHDocVw.DWebBrowserEvents2_DocumentCompleteEvent) Handles wb.DocumentComplete
Dim ele As IHTMLElement
Dim node As TreeNode
Doc3 = wb.Document
tvDOM.Nodes.Clear()
ele = Doc3.documentElement
node = tvDOM.Nodes.Add(ele.tagName + "_" + ele.sourceIndex.ToString)
node.Tag = ele.sourceIndex
Call add_Children(ele, node)
......... 'the parsing program
If rsPage.Read() Then
PageURL = rsPage.Item("PageURL")
PageID = rsPage.Item("PageID")
wb.Navigate(PageURL)
Doc3 = wb.Document
Else
cn.Close()
rsPage.Close()
rsPage = Nothing
cmad = Nothing
cn = Nothing
End If
End Sub
wb: AxSHDocVw.AxWebBrowser
Anyone find a solution to documentcompleted event?
cplusplus1
I don't have any error message for the parsing error.
I find it from the output data
I had post the error output data in the last message
Y. J.
SPWilkinson
Ondra Br.
This situation was not happened everytime.
Sometime it will happen.
I analysis the log data below.
The different two PageURL-34,35, but both of two web page have the same DOM tree structure.
So I guess that it didn't download complete for some reasons.
I read the technical reports for this event, and I know that this event will be trigger after the HTML document download complete.
Any method I can use to detect the wrong situation
Thanks a lot!
Y. J.
log data:
PageID PageURL ImageID ImageFileName ImageLinkWeight OutLinkPageID OutLinkPageURL
34 http://creampuffsinvenice.typepad.com/ 34-101 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2250_1.jpg 5205 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2250_1.jpg
34 http://creampuffsinvenice.typepad.com/ 34-245 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2135.jpg 5206 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2135.jpg
34 http://creampuffsinvenice.typepad.com/ 34-310 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2115.jpg 5207 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2115.jpg
34 http://creampuffsinvenice.typepad.com/ 34-33 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2148.jpg 5203 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2148.jpg
34 http://creampuffsinvenice.typepad.com/ 34-352 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2103.jpg 5208 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2103.jpg
34 http://creampuffsinvenice.typepad.com/ 34-369 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2083_1.jpg 5209 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2083_1.jpg
34 http://creampuffsinvenice.typepad.com/ 34-392 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2073.jpg 5210 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2073.jpg
34 http://creampuffsinvenice.typepad.com/ 34-461 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2052.jpg 5211 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2052.jpg
34 http://creampuffsinvenice.typepad.com/ 34-480 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2048.jpg 5212 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2048.jpg
34 http://creampuffsinvenice.typepad.com/ 34-539 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn1909.jpg 5213 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn1909.jpg
34 http://creampuffsinvenice.typepad.com/ 34-549 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn1902_1.jpg 5214 http://creampuffsinvenice.typepad.com/.shared/image.html /photos/uncategorized/dscn1902_1.jpg
34 http://creampuffsinvenice.typepad.com/ 34-561 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn1911_1.jpg 5215 http://creampuffsinvenice.typepad.com/.shared/image.html /photos/uncategorized/dscn1911_1.jpg
34 http://creampuffsinvenice.typepad.com/ 34-63 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2163.jpg 5204 http://creampuffsinvenice.typepad.com/.shared/image.html /photos/uncategorized/dscn2163.jpg
34 http://creampuffsinvenice.typepad.com/ 34-639 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2027.jpg 5216 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2027.jpg
34 http://creampuffsinvenice.typepad.com/ 34-667 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2044.jpg 5217 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2044.jpg
34 http://creampuffsinvenice.typepad.com/ 34-769 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2025.jpg 5218 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2025.jpg
35 http://www.eurohotelguide.com/venice/ 35-101 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2250_1.jpg 5205 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2250_1.jpg
35 http://www.eurohotelguide.com/venice/ 35-245 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2135.jpg 5206 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2135.jpg
35 http://www.eurohotelguide.com/venice/ 35-310 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2115.jpg 5207 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2115.jpg
35 http://www.eurohotelguide.com/venice/ 35-33 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2148.jpg 5203 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2148.jpg
35 http://www.eurohotelguide.com/venice/ 35-352 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2103.jpg 5208 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2103.jpg
35 http://www.eurohotelguide.com/venice/ 35-369 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2083_1.jpg 5209 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2083_1.jpg
35 http://www.eurohotelguide.com/venice/ 35-392 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2073.jpg 5210 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2073.jpg
35 http://www.eurohotelguide.com/venice/ 35-461 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2052.jpg 5211 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2052.jpg
35 http://www.eurohotelguide.com/venice/ 35-480 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2048.jpg 5212 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2048.jpg
35 http://www.eurohotelguide.com/venice/ 35-539 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn1909.jpg 5213 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn1909.jpg
35 http://www.eurohotelguide.com/venice/ 35-549 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn1902_1.jpg 5214 http://creampuffsinvenice.typepad.com/.shared/image.html /photos/uncategorized/dscn1902_1.jpg
35 http://www.eurohotelguide.com/venice/ 35-561 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn1911_1.jpg 5215 http://creampuffsinvenice.typepad.com/.shared/image.html /photos/uncategorized/dscn1911_1.jpg
35 http://www.eurohotelguide.com/venice/ 35-63 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2163.jpg 5204 http://creampuffsinvenice.typepad.com/.shared/image.html /photos/uncategorized/dscn2163.jpg
35 http://www.eurohotelguide.com/venice/ 35-639 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2027.jpg 5216 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2027.jpg
35 http://www.eurohotelguide.com/venice/ 35-667 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2044.jpg 5217 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2044.jpg
35 http://www.eurohotelguide.com/venice/ 35-769 http://creampuffsinvenice.typepad.com/cream_puffs_in_venice/images/dscn2025.jpg 5218 http://creampuffsinvenice.typepad.com/photos/uncategorized/dscn2025.jpg
Ceds
Very good posting,
Indeed I applied the algoritm provided and I got my application worked fine.
Remember the algorithm:
1. Load/Refresh page, wait for pc00.
2. Wait for a little bit (0.25-1.0sec), to see if any Navigated, Navigating, or ProgressChanged events fire. If they fire, then goto 1. Otherwise goto 3.
3. Consider the web page loaded completely.
I used C# and below is the code implemented:
//global variable declaration for the doc status
private bool customdocumentcompleted = false;
//screenscraping is my webbrowser control
//add the below events on the web browser control-> right click properties -> Events (small tool box on the top of properties)
//document completed is the default status but since the page contains many frames
//the status can be completed but maybe is not true, so a double-check is needed.
private void screenscraping_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
//Double-check completed status.
//any time when an extra confirmation on doc completed needed please include the below two lines.
while (customdocumentcompleted == false){
//loop to allow customdocumentcompleted = true
// not 0.25 or 1 seconds are required since the loop will take care of it.
System.Windows.Forms.Application.DoEvents();
}
//next step I want to perform after the document is complete.
//You can include a switch or case statement to do many steps
// .....
//
//
}
private void screenscraping_Navigated(object sender, WebBrowserNavigatedEventArgs e)
{
// update my custom "completed" status
updateBrowserStatus(String.Empty);
}
private void screenscraping_Navigating(object sender, WebBrowserNavigatingEventArgs e)
{
// Provide feedback to user that the browser is busy
updateBrowserStatus("Navigating...");
}
private void screenscraping_ProgressChanged(object sender, WebBrowserProgressChangedEventArgs e)
{
customdocumentcompleted = false;
//check if the document is truly completed. rickp
if (e.CurrentProgress == 0 && e.MaximumProgress == 0)
{
//only here the status completed is updated to true.!!!
customdocumentcompleted = true;
string message = "Ready";
toolStripStatusLabel2.Text = message;
}
}
// set customdocumentcompleted to false because the page still being processed
private void updateBrowserStatus(string message)
{
customdocumentcompleted = false;
toolStripStatusLabel2.Text = message; //label saying the page is navigating...up to you.
}
Thanks,
ricpue@gmail.com
SB1
thanks for your help!
I modified my program.
It can download the web pages most of time.
yjtsai
Phil D.
Can you get a stack trace at the time when your parsing fails If you're not getting an exception how do you detect that the parsing fails
I took a look at that site and it looks like it's contstantly updated (several adds). Could the changing adds be the source of your problem
zand108
Notes:
1. WebBrowser.ProgressChanged is fired pretty much every time the WebBrowser finishes loading all pages in its HtmlWindow (including sub-documents). When this is called, WebBrowserProgressChangedEventArgs.CurrentProgress == 0 and WebBrowserProgressChangedEventArgs.MaximumProgress == 0 (which I will call pc00 from now on).
2. Some pages have javascript that wait for the page to load, and then they load an internal frame. This results in pc00 being fired twice.
3. Even refreshes finish firing pc00, which is a great since DocumentCompleted isn't fired.
4. Sometimes DocumentCompleted isn't fired.
Algorithm:
1. Load/Refresh page, wait for pc00.
2. Wait for a little bit (0.25-1.0sec), to see if any Navigated, Navigating, or ProgressChanged events fire. If they fire, then goto 1. Otherwise goto 3.
3. Consider the web page loaded completely.
Data:
ng=Navigating
nd=Navigated
pc=ProgressChanged
dc=DocumentCompleted
LOAD
===========================
ng:http://w3schools.com/html/tryit.asp filename=tryhtml_imglink
nd:http://w3schools.com/html/tryit.asp filename=tryhtml_imglink
ng:http://w3schools.com/banners/bannerframe.asp adpartner=aspall2
ng:http://w3schools.com/html/tryit_view.asp filename=tryhtml_imglinkview
pc:0 / 10000
pc:0 / 10000
ng:http://ads.aspalliance.com/displayad.aspx t=13&m=43&guid=1
nd:http://w3schools.com/banners/bannerframe.asp adpartner=aspall2
nd:http://w3schools.com/html/tryit_view.asp filename=tryhtml_imglink
pc:10000 / 10000
pc:-1 / 10000
dc:http://w3schools.com/html/tryit_view.asp filename=tryhtml_imglink
pc:10000 / 10000
nd:http://ads.aspalliance.com/displayad.aspx t=13&m=43&guid=1
pc:1097700 / 1000000
pc:1130900 / 1000000
pc:1951000 / 1000000
pc:2000000 / 1000000
pc:-1 / 1000000
dc:http://ads.aspalliance.com/displayad.aspx t=13&m=43&guid=1, c=System.Windows.Forms.WebBrowser
dc:http://w3schools.com/banners/bannerframe.asp adpartner=aspall2
pc:2000000 / 1000000
dc:http://w3schools.com/html/tryit.asp filename=tryhtml_imglink
pc:0 / 0
LOAD
===========================
ng:http://rebirthofreason.com/
pc:100 / 10000
pc:100 / 10000
pc:150 / 10000
pc:200 / 10000
pc:37500 / 1000000
nd:http://rebirthofreason.com/
pc:134300 / 1000000
pc:966000 / 1000000
pc:977700 / 1000000
pc:1000000 / 1000000
pc:-1 / 1000000
pc:1000000 / 1000000
dc:http://rebirthofreason.com/
pc:0 / 0
LOAD
===========================
ng:http://rebirthofreason.com/cgi-bin/SHQ/SHQ_SoloMail2.cgi Function=EditDraft&maiID=0
pc:0 / 10000
pc:50 / 10000
pc:444200 / 1000000
nd:http://rebirthofreason.com/cgi-bin/SHQ/SHQ_SoloMail2.cgi Function=EditDraft&maiID=0
pc:1000000 / 1000000
pc:-1 / 1000000
pc:1000000 / 1000000
dc:http://rebirthofreason.com/cgi-bin/SHQ/SHQ_SoloMail2.cgi Function=EditDraft&maiID=0
ng:http://rebirthofreason.com/tiny_mce/blank.htm
pc:-1 / 1000000
pc:0 / 0
pc:12200 / 1000000
pc:1000000 / 1000000
pc:-1 / 1000000
pc:0 / 0
RELOAD
===========================
pc:34900 / 1000000
pc:86200 / 1000000
pc:685800 / 1000000
pc:1000000 / 1000000
pc:-1 / 1000000
pc:1000000 / 1000000
ng:http://rebirthofreason.com/tiny_mce/blank.htm
pc:-1 / 1000000
pc:0 / 0
LOAD
===========================
ng:http://mail.yahoo.com/
pc:100 / 10000
pc:100 / 10000
pc:100 / 10000
pc:100 / 10000
nd:https://login.yahoo.com/config/login_verify2 &.src=ym
pc:150 / 10000
pc:200 / 10000
pc:10000 / 10000
pc:-1 / 10000
pc:10000 / 10000
dc:https://login.yahoo.com/config/login_verify2 &.src=ym
pc:0 / 0
LOAD
===========================
ng:https://login.yahoo.com/config/login
pc:0 / 10000
nd:https://login.yahoo.com/config/login
ng:http://us.f584.mail.yahoo.com/ym/login .rand=76pk0qrib09hf
pc:100 / 10000
pc:100 / 10000
pc:150 / 10000
nd:http://us.f584.mail.yahoo.com/ym/login .rand=76pk0qrib09hf
ng:about:blankygmaprocframe
nd:about:blank
pc:10000 / 10000
pc:-1 / 10000
dc:about:blank
pc:0 / 0
pc:11300 / 1000000
pc:22600 / 1000000
pc:32600 / 1000000
pc:963000 / 1000000
pc:978700 / 1000000
ng:about:blankasynch
nd:about:blank
pc:1000000 / 1000000
pc:-1 / 1000000
dc:about:blank
pc:1000000 / 1000000
dc:http://us.f584.mail.yahoo.com/ym/login .rand=76pk0qrib09hf
pc:0 / 0
LOAD
===========================
ng:http://us.f584.mail.yahoo.com/ym/Compose YY=29481&y5beta=yes
pc:0 / 10000
nd:http://us.f584.mail.yahoo.com/ym/Compose YY=29481&y5beta=yes
ng:about:blankspaceidframe
ng:about:blankygmaprocframe
nd:about:blank
pc:10000 / 10000
pc:-1 / 10000
dc:about:blank
nd:about:blank
dc:about:blank
ng:about:blank
ng:about:blank
pc:-1 / 10000
nd:about:blank
dc:about:blank
ng:about:blankasynch
nd:about:blank
pc:-1 / 10000
dc:about:blank
pc:10000 / 10000
dc:http://us.f584.mail.yahoo.com/ym/Compose YY=29481&y5beta=yes
pc:0 / 0
Vivek S
Just to clarify - from numerous tests, the documentcompleted event DOES fire for each IFRAME on a page. This ensures that programmers can not tell when the page is really done! In my tests on http://www.foxnews.com it fires 7 or 8 times!
Also, the documentcompleted event and documentcomplete event are DIFFERENT. documentcompleted is NET 2.0 and documentcomplete is from NET 1 and VB6...from my understanding at least.
In either case, there is no workable solution I have seen. While I appreciate the assistance from the moderators, as I know you all are very knowledgeable, I do wish you would perform a test so you can see the documentcompleted fires more than once per page...which is NOT what the Microsoft documentation says it should do here:
http://msdn2.microsoft.com/en-us/library/system.windows.forms.webbrowser.documentcompleted.aspx
QUOTE FROM PAGE: "When the DocumentCompleted event occurs, the new document is fully loaded" <-NOT TRUE!
If you need a sample app to demonstrate, reply to this post and I will post it for you to see.
I say this is a BUG.